Transport Tycoon Forums

The place to talk about Transport Tycoon
It is currently Fri Apr 19, 2019 9:15 pm

All times are UTC




Post new topic  Reply to topic  [ 22 posts ]  Go to page 1 2 Next
Author Message
PostPosted: Fri Dec 13, 2013 10:03 am 
Offline
Engineer
Engineer

Joined: Tue Jun 18, 2013 6:18 pm
Posts: 13
Hello,

** Long version **
I'm a linux user (and I contribute to Wayland).
I'm a fan of OpenTTD.

Currently, OpenTTD under Linux talks to the X server via SDL 1.2.
The blitter uses RAM memory to draw the content, and then asks to SDL1.2 to display the RAM content.
For that, SDL_UpdateRects is called: it'll make the user see the changes, and OpenTTD indicates the damaged regions.

SDL 1.2 on its side, uses XShmPutImage or XPutImage on each damaged rect to give the new content to the X server,
and the X DDX (what cares of graphic acceleration, etc) handles the PutImage call.

Glamor is a library used optionally by some DDX, and is the default on some others (recent radeon cards, and XWayland, the X layer under Wayland).
Glamor stores pixmaps into GL texture.
Each PutImage call will result into a conversion of RAM memory into GL texture.
As you may know, this isn't usually a fast operation.
That makes OpenTTD really laggy with DDXs using Glamor.

A short fix is to change MAX_DIRTY_RECTS to a really low value:
performance becomes ok again with Glamor, because we do only one buffer -> gl texture conversion, instead of many ones.
However in other situations, we wouldn't like MAX_DIRTY_RECTS to be a low value.

Since SDL2 brings some new functionalities that some would like to see, as the possibility to have several windows at a time,
then I suggest the SDL 1.2 code should be ported to SDL2.
SDL1.2 SDL_UpdateRects calls becomes SDL2 calls to SDL_UpdateWindowSurfaceRects
The difference is that SDL_UpdateWindowSurfaceRects is clever, and in some cases (currently it seems to be when using an Amd card or an Nvidia card),
it avoids the PutImage calls, and instead will get a big rect containing the small damaged rects, and convert the buffer content of this rect into gl texture.
It'll then commit the gl texture.

So using SDL2 fully replaces the short fix.
** Short version **
Using SDL2 would bring performance benefits in some situations.

I'm not an SDL or SDL2 programmer, but it looks like the porting shouldn't be too hard.
SDL2 spirit is to use SDL_Texture, but I think in OpenTTD specific case, it should not using it with the current blitter (a gl blitter could use it).
I think OpenTTD should use SDL_UpdateWindowSurfaceRects. It looks like all the other changes becomes some renaming, but as I said, I'm not an expert.


Top
   
PostPosted: Fri Dec 13, 2013 12:39 pm 
Offline
Tycoon
Tycoon
User avatar

Joined: Thu Jun 25, 2009 4:42 pm
Posts: 2365
Location: Location, Location
I brought this up to the devs a couple of months ago. SDL2 will be used, but at some point in the future, as OTTD has to think about compatiblity with other OSs which don't currently have easy access to SDL2 (Past versions of Ubuntu, for example)

_________________
AroAI - A really feeble attempt at an AI

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra


Top
   
PostPosted: Fri Dec 13, 2013 2:04 pm 
Offline
Engineer
Engineer

Joined: Tue Jun 18, 2013 6:18 pm
Posts: 13
I the changes are not too heavy,
there could be both sdl 1.2 and sdl 2.0 code, protected by defines.
When compiling it would detect if sdl 2 is present when running configure,
and the code would use either the sdl 1.2 path or the sdl 2.0 path.


Top
   
PostPosted: Fri Dec 13, 2013 7:38 pm 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Thu Feb 09, 2006 7:15 pm
Posts: 3815
Looking at the migration page and certain APIs I think it is significantly more than just a few #ifdefs.

SDL_SetColors used to get a SDL_Surface, but now SDL_SetPaletteColors needs a SDL_Palette.
SDL_BlitSurface used to get two SDL_Surfaces and two SDL_Rects, but now that needs to become SDL_RenderCopy with a SDL_Renderer, SDL_Texture and two SDL_Rects.
SDL_GetKeyState used to return KeySyms, but now SDL_GetKeyboardState returns ScanCodes.

I'd reckon that creating a new SDL2 video driver will result in easier to read code, and less problems overall. It will also allow one to make more use of the newer features of SDL2.


Top
   
PostPosted: Fri Dec 13, 2013 8:26 pm 
Offline
Engineer
Engineer

Joined: Tue Jun 18, 2013 6:18 pm
Posts: 13
SDL_BlitSurface still exists.

I get the impression that the documentation to port SDL 1.2 to SDL 2.0 doesn't apply well for OpenTTD case.

Since OpenTTD current blitter is entirely software, I think it should not use SDL renderer or SDL textures at all. That would decrease performance to use textures at this step, instead at the end, in SDL_UpdateWindowSurfaceRects.

Except if you plan to use a blitter using the SDL rendering functions, I advise to keep near to the current code, and call the similar functions of the new API (and then avoid SDL renderers and SDL textures).


Top
   
PostPosted: Fri Dec 13, 2013 10:55 pm 
Offline
Chief Executive
Chief Executive
User avatar

Joined: Mon Nov 15, 2004 7:46 pm
Posts: 645
Location: Berlin, Germany
We made some tests for simutrans (which uses a more or less similar software rendering) and SDL versus SDL2 backend. Conversion was not too difficult. A subtle issues: SDL2 cannot use sounds with different sampling rates together.

SDL2 is certainly not faster but at least about 20% slower in most cases. The exception is the newest MAC OS, where only SDL2 gets a hardware acceleration (which is mandantory with a retina display). Otherwise screen rendering hits the bandwidth of the memory transfer the CPU can do.

See performance of SDL2 on windows with an otherwise identical game to display (GDI is just plain old windows routines, not DirectX)
Code:
SDL2   13.6 ms/frame
OpenGL, PBO   9.3 ms/frame
SDL1, ST, -use_hw   9.0 ms/frame
OpenGL   8.3 ms/frame
GDI, ST   7.6 ms/frame
SDL1, ST, UpdateRect   7.4 ms/frame
SDL1, ST, UpdateRects   6.5 ms/frame
GDI, MT   5.4 ms/frame
SDL1, MT   5.3 ms/frame
ST=single threaded   MT=multi threade


At least for 16 bit/pixel bitmaps, SDL2 is is half speed of single threaded SDL1.2 ... So the majority (i.e. the windows users) might get rather a speed loss than a gain. (Of course with 32 bit bitmaps, it may be less obvious, since these come even closer to the bandwidth bottleneck). But I am pretty sure, 8 bit bitmaps will be better in SDL1.2 too, since OpenGL does not really care about them any more.

As noted above, SDL2 really needs merging of dirty rects as much as possible. THe above number are with the optimized algorithm of getting as few as possible dirty rects.

_________________
I like to look at great maps and see how things flow. A little like a finished model railway, but it is evolving and actually never finished. http://www.simutrans.com


Top
   
PostPosted: Sat Dec 14, 2013 9:49 am 
Offline
Engineer
Engineer

Joined: Tue Jun 18, 2013 6:18 pm
Posts: 13
Are you sure it isn't a vsync issue?

When reading the code,
I read that
in your SDL1 code: async_blit = parameter[0];
SDL2 code: sync_blit = parameter[0];

Perhaps if you try sync_blit = !parameter [0] you'll get better results...


Even if it's not that, the use of SDL of OpenTTD looks very different to yours, so I won't expect similar results.


Top
   
PostPosted: Sat Dec 14, 2013 9:32 pm 
Offline
Chief Executive
Chief Executive
User avatar

Joined: Mon Nov 15, 2004 7:46 pm
Posts: 645
Location: Berlin, Germany
While the code is different, the general action first rendering to buffer and then copy the dirty areas only to screen are similar. OpenTTD has the added difficulty of a more fixed frame rate, i.e. could not run with 10 fps and similar timing compared to simutrans (or that was my knowledge last time I deeply looked at the OpenTTD code).

I will check for the vsync issues but this was not the only bottleneck. I saw that OpenTTD does not make any attempt to merge dirty rects, which is the key with SDL2. Otherwise SDL2 was even slower (with about 50% more dirty rect covering the same area, SDL2 needed about twice the time for the copy per frame). Copying the whole frame each update was also tried, but with resolutions of fullHD and even more this was very slow. Considering that mobile phones already have such resolutions, together with CPU power (and memory to screen bandwidth) much less than a desktop this becomes even more important.

Finally threading for copying was even slower with SDL2. Speculation was that the hardware blitter could not cope well and one had a queue which was just filled with additional threading overhead.

While SDL2 hardware support is certainly nice to have, oldschool games which did their rendering before in anything other than 32 bit seems a little neglected. Otherwise I see no reason which a portable library is not improving on the platform at least 80% of user are using ...

BTW: I still need to check this, but if SDL2 does not have Unicode composed charecters, then any asian splayer will not be happy with SDL2 ...

_________________
I like to look at great maps and see how things flow. A little like a finished model railway, but it is evolving and actually never finished. http://www.simutrans.com


Top
   
PostPosted: Tue Dec 17, 2013 9:55 pm 
Offline
Engineer
Engineer

Joined: Tue Jun 18, 2013 6:18 pm
Posts: 13
I've looked at SDL2 and SDL1.2 code to see if SDL_UpdateWindowSurfaceRects would behave differently (performance-wise) than SDL_UpdateRects for Windows.

It appears that for SDL 1.2, it used to call BitBlt for every damaged rect, while now it calls BitBlt one time, but on the whole screen.

I don't know Windows internal, so I can't say if for OpenTTD it would be a performance benefit or not. I just imagine that this change has been done for a good reason.

prissi: Just out of curiosity, does your fps changes with the change I mentionned? (Was Vsync the problem?) I don't know from where comes your performance loss. But I doubt OpenTTD would be affected, since it doesn't use hardware acceleration to render.


Top
   
PostPosted: Tue Dec 17, 2013 10:26 pm 
Offline
Chief Executive
Chief Executive
User avatar

Joined: Mon Nov 15, 2004 7:46 pm
Posts: 645
Location: Berlin, Germany
Simutrans does not use hardware acceleration either. But it seems that SDL can use OpenGL hardware acceleration directly (and hence the one big copy).

In the profiling thread in the forum SDL2 got slower and slower the higher the number of dirty rectangles to update. However, full update was even slower. So it may be something like deferred copy to screen and blocking all calls until the last has not been processed or so. Also locking/not-locking the surface was very important.

All things point towards a clash with the new architecture rather than simply some wrong settings in the initialisation. Nevertheless, at OpenTTD 8 or 32 bit depth the situation might be better. (But then OpennTTD already uses DirectX, if memory serves me right.)

_________________
I like to look at great maps and see how things flow. A little like a finished model railway, but it is evolving and actually never finished. http://www.simutrans.com


Top
   
PostPosted: Tue Dec 17, 2013 10:49 pm 
Offline
Tycoon
Tycoon

Joined: Wed Jan 17, 2007 12:14 am
Posts: 7312
prissi wrote:
(But then OpennTTD already uses DirectX, if memory serves me right.)

OpenTTD doesn't use SDL for windows (by default, can be compiled in afair), but i think it uses GDI, not DirectX

_________________
You might not exactly be interested in Ferion, but if you are, have fun :)


Top
   
PostPosted: Thu Dec 19, 2013 10:54 am 
Offline
Engineer
Engineer

Joined: Tue Jun 18, 2013 6:18 pm
Posts: 13
Rubidium wrote:
Looking at the migration page and certain APIs I think it is significantly more than just a few #ifdefs.

SDL_SetColors used to get a SDL_Surface, but now SDL_SetPaletteColors needs a SDL_Palette.
SDL_BlitSurface used to get two SDL_Surfaces and two SDL_Rects, but now that needs to become SDL_RenderCopy with a SDL_Renderer, SDL_Texture and two SDL_Rects.
SDL_GetKeyState used to return KeySyms, but now SDL_GetKeyboardState returns ScanCodes.

I'd reckon that creating a new SDL2 video driver will result in easier to read code, and less problems overall. It will also allow one to make more use of the newer features of SDL2.


I've checked again the source code, and figured out that :
. SDL_UpdateWindowSurfaceRects is equivalent to SDL_UpdateRects as I said.

. Some platforms specify a particular optimized behaviour for it (as for SDL_UpdateRects)

. But as I said in some situations it will prefer using GL textures, and use GL to commit the window content.
In this situation, it'll do exactly the same thing that what is describe in the SDL1.2 -> SDL2 tutorial (part "If your game just wants to get fully-rendered frames to the screen").

. SDL_BlitSurface does still exist.

. SDL_GetKeyFromScancode can convert a scancode given by SDL_GetKeyboardState to a SDL_Keycode. SDL_GetKeyName can get a utf-8 character from it. If it helps replace SDL_GetKeyState...


Top
   
PostPosted: Sun Oct 02, 2016 9:44 am 
Offline
Engineer
Engineer

Joined: Wed Sep 27, 2006 6:04 pm
Posts: 77
Is anybody still considering adding support for SDL 2, either by porting the SDL 1 code, or by adding a separate video driver?

Currently there is no support for pasting from the clipboard when compiling on UNIX systems, which means that long names such as yapf.rail_firstred_twoway_eol have to be typed at the console. SDL 2 does support pasting, and on the other hand adding support for X11 clipboard bypassing SDL would be a huge mess.


Top
   
PostPosted: Sun Oct 02, 2016 10:08 am 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4707
Location: home
lcd_47 wrote:
Is anybody still considering adding support for SDL 2, either by porting the SDL 1 code, or by adding a separate video driver?
Is 3 years of silence not a sufficient answer to your question?

_________________
Being a OpenTTD developer does not mean I know what I am doing.
Also, other OpenTTD developers may have different opinions.


Top
   
PostPosted: Sun Oct 02, 2016 10:16 am 
Offline
Engineer
Engineer

Joined: Wed Sep 27, 2006 6:04 pm
Posts: 77
Alberth wrote:
Is 3 years of silence not a sufficient answer to your question?

Is my question not enough of a hint that I'm actually trying to politely suggest changing the answer?


Top
   
PostPosted: Mon Oct 03, 2016 4:03 pm 
Offline
Tycoon
Tycoon

Joined: Wed Jan 17, 2007 12:14 am
Posts: 7312
well, nobody prevents you from changing the answer by working on it...

_________________
You might not exactly be interested in Ferion, but if you are, have fun :)


Top
   
PostPosted: Mon Oct 03, 2016 4:26 pm 
Offline
Engineer
Engineer

Joined: Wed Sep 27, 2006 6:04 pm
Posts: 77
Eddi wrote:
well, nobody prevents you from changing the answer by working on it...

Not my itch, sorry. My itch (that I might scratch at some point) is that copying from clipboard doesn't work on UNIX. The "huge mess" solution that bypasses SDL would be good enough for solving that. :D


Top
   
PostPosted: Mon Oct 03, 2016 4:42 pm 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4707
Location: home
Fix that, and as a side effect, port to SDL2 ;)

_________________
Being a OpenTTD developer does not mean I know what I am doing.
Also, other OpenTTD developers may have different opinions.


Top
   
PostPosted: Sun Jan 20, 2019 6:35 am 
Offline
Engineer
Engineer
User avatar

Joined: Fri Jan 11, 2019 3:03 am
Posts: 11
Location: New York, NY
Hey, so I've been working on this over the past few days. I've gotten OpenTTD's SDL driver working on SDL 2 on this branch: https://github.com/nikolas/OpenTTD/tree/sdl2 I'm probably going to clean this up, fix up loose ends and make a pull request at some point.


Top
   
PostPosted: Mon Jan 28, 2019 4:12 pm 
Offline
Engineer
Engineer
User avatar

Joined: Fri Jan 11, 2019 3:03 am
Posts: 11
Location: New York, NY
There was some discussion about dirty rectangles in this thread. Here's a relevant part from the SDL2 MigrationGuide: https://wiki.libsdl.org/MigrationGuide

Quote:
At the end of the frame, we want to upload to the texture like this:

SDL_UpdateTexture(sdlTexture, NULL, myPixels, 640 * sizeof (Uint32));

This will upload your pixels to GPU memory. That NULL can be a subregion if you want to mess around with dirty rectangles, but chances are modern hardware can just swallow the whole framebuffer without much trouble.


So.. that's exactly what I've done in my SDL2 pull request, here: https://github.com/OpenTTD/OpenTTD/pull/7086/files#diff-c43e36860453567981f521d129323763L158

The dirty rect system is already in place in OpenTTD's SDL video driver, so it couldn't hurt to use it. But when I updated parts of the screen in a for loop like the 1.2 driver does, I saw some intense flickering. Though that may be avoided somehow. But I haven't seen any problems with this entire screen update, and I'm all for simple solutions that just work.

Also, as I've noted in the pull request (https://github.com/OpenTTD/OpenTTD/pull/7086), I'm specifying the SDL 2 renderer as a software renderer for stability reasons. Messing with the GPU sounds nice but introduces some problems that may not work on everyone's setups.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 22 posts ]  Go to page 1 2 Next

All times are UTC


Who is online

Users browsing this forum: Google [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000-2019 phpBB Limited

Copyright © Owen Rudge/The Transport Tycoon Forums 2001-2019.
Hosted by Zernebok Hosting.