Hello,
** Long version **
I'm a linux user (and I contribute to Wayland).
I'm a fan of OpenTTD.
Currently, OpenTTD under Linux talks to the X server via SDL 1.2.
The blitter uses RAM memory to draw the content, and then asks to SDL1.2 to display the RAM content.
For that, SDL_UpdateRects is called: it'll make the user see the changes, and OpenTTD indicates the damaged regions.
SDL 1.2 on its side, uses XShmPutImage or XPutImage on each damaged rect to give the new content to the X server,
and the X DDX (what cares of graphic acceleration, etc) handles the PutImage call.
Glamor is a library used optionally by some DDX, and is the default on some others (recent radeon cards, and XWayland, the X layer under Wayland).
Glamor stores pixmaps into GL texture.
Each PutImage call will result into a conversion of RAM memory into GL texture.
As you may know, this isn't usually a fast operation.
That makes OpenTTD really laggy with DDXs using Glamor.
A short fix is to change MAX_DIRTY_RECTS to a really low value:
performance becomes ok again with Glamor, because we do only one buffer -> gl texture conversion, instead of many ones.
However in other situations, we wouldn't like MAX_DIRTY_RECTS to be a low value.
Since SDL2 brings some new functionalities that some would like to see, as the possibility to have several windows at a time,
then I suggest the SDL 1.2 code should be ported to SDL2.
SDL1.2 SDL_UpdateRects calls becomes SDL2 calls to SDL_UpdateWindowSurfaceRects
The difference is that SDL_UpdateWindowSurfaceRects is clever, and in some cases (currently it seems to be when using an Amd card or an Nvidia card),
it avoids the PutImage calls, and instead will get a big rect containing the small damaged rects, and convert the buffer content of this rect into gl texture.
It'll then commit the gl texture.
So using SDL2 fully replaces the short fix.
** Short version **
Using SDL2 would bring performance benefits in some situations.
I'm not an SDL or SDL2 programmer, but it looks like the porting shouldn't be too hard.
SDL2 spirit is to use SDL_Texture, but I think in OpenTTD specific case, it should not using it with the current blitter (a gl blitter could use it).
I think OpenTTD should use SDL_UpdateWindowSurfaceRects. It looks like all the other changes becomes some renaming, but as I said, I'm not an expert.
Port SDL 1.2 code to SDL 2.0
Moderator: OpenTTD Developers
Re: Port SDL 1.2 code to SDL 2.0
I brought this up to the devs a couple of months ago. SDL2 will be used, but at some point in the future, as OTTD has to think about compatiblity with other OSs which don't currently have easy access to SDL2 (Past versions of Ubuntu, for example)
AroAI - A really feeble attempt at an AI
It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra
It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra
Re: Port SDL 1.2 code to SDL 2.0
I the changes are not too heavy,
there could be both sdl 1.2 and sdl 2.0 code, protected by defines.
When compiling it would detect if sdl 2 is present when running configure,
and the code would use either the sdl 1.2 path or the sdl 2.0 path.
there could be both sdl 1.2 and sdl 2.0 code, protected by defines.
When compiling it would detect if sdl 2 is present when running configure,
and the code would use either the sdl 1.2 path or the sdl 2.0 path.
Re: Port SDL 1.2 code to SDL 2.0
Looking at the migration page and certain APIs I think it is significantly more than just a few #ifdefs.
SDL_SetColors used to get a SDL_Surface, but now SDL_SetPaletteColors needs a SDL_Palette.
SDL_BlitSurface used to get two SDL_Surfaces and two SDL_Rects, but now that needs to become SDL_RenderCopy with a SDL_Renderer, SDL_Texture and two SDL_Rects.
SDL_GetKeyState used to return KeySyms, but now SDL_GetKeyboardState returns ScanCodes.
I'd reckon that creating a new SDL2 video driver will result in easier to read code, and less problems overall. It will also allow one to make more use of the newer features of SDL2.
SDL_SetColors used to get a SDL_Surface, but now SDL_SetPaletteColors needs a SDL_Palette.
SDL_BlitSurface used to get two SDL_Surfaces and two SDL_Rects, but now that needs to become SDL_RenderCopy with a SDL_Renderer, SDL_Texture and two SDL_Rects.
SDL_GetKeyState used to return KeySyms, but now SDL_GetKeyboardState returns ScanCodes.
I'd reckon that creating a new SDL2 video driver will result in easier to read code, and less problems overall. It will also allow one to make more use of the newer features of SDL2.
Re: Port SDL 1.2 code to SDL 2.0
SDL_BlitSurface still exists.
I get the impression that the documentation to port SDL 1.2 to SDL 2.0 doesn't apply well for OpenTTD case.
Since OpenTTD current blitter is entirely software, I think it should not use SDL renderer or SDL textures at all. That would decrease performance to use textures at this step, instead at the end, in SDL_UpdateWindowSurfaceRects.
Except if you plan to use a blitter using the SDL rendering functions, I advise to keep near to the current code, and call the similar functions of the new API (and then avoid SDL renderers and SDL textures).
I get the impression that the documentation to port SDL 1.2 to SDL 2.0 doesn't apply well for OpenTTD case.
Since OpenTTD current blitter is entirely software, I think it should not use SDL renderer or SDL textures at all. That would decrease performance to use textures at this step, instead at the end, in SDL_UpdateWindowSurfaceRects.
Except if you plan to use a blitter using the SDL rendering functions, I advise to keep near to the current code, and call the similar functions of the new API (and then avoid SDL renderers and SDL textures).
Re: Port SDL 1.2 code to SDL 2.0
We made some tests for simutrans (which uses a more or less similar software rendering) and SDL versus SDL2 backend. Conversion was not too difficult. A subtle issues: SDL2 cannot use sounds with different sampling rates together.
SDL2 is certainly not faster but at least about 20% slower in most cases. The exception is the newest MAC OS, where only SDL2 gets a hardware acceleration (which is mandantory with a retina display). Otherwise screen rendering hits the bandwidth of the memory transfer the CPU can do.
See performance of SDL2 on windows with an otherwise identical game to display (GDI is just plain old windows routines, not DirectX)
At least for 16 bit/pixel bitmaps, SDL2 is is half speed of single threaded SDL1.2 ... So the majority (i.e. the windows users) might get rather a speed loss than a gain. (Of course with 32 bit bitmaps, it may be less obvious, since these come even closer to the bandwidth bottleneck). But I am pretty sure, 8 bit bitmaps will be better in SDL1.2 too, since OpenGL does not really care about them any more.
As noted above, SDL2 really needs merging of dirty rects as much as possible. THe above number are with the optimized algorithm of getting as few as possible dirty rects.
SDL2 is certainly not faster but at least about 20% slower in most cases. The exception is the newest MAC OS, where only SDL2 gets a hardware acceleration (which is mandantory with a retina display). Otherwise screen rendering hits the bandwidth of the memory transfer the CPU can do.
See performance of SDL2 on windows with an otherwise identical game to display (GDI is just plain old windows routines, not DirectX)
Code: Select all
SDL2 13.6 ms/frame
OpenGL, PBO 9.3 ms/frame
SDL1, ST, -use_hw 9.0 ms/frame
OpenGL 8.3 ms/frame
GDI, ST 7.6 ms/frame
SDL1, ST, UpdateRect 7.4 ms/frame
SDL1, ST, UpdateRects 6.5 ms/frame
GDI, MT 5.4 ms/frame
SDL1, MT 5.3 ms/frame
ST=single threaded MT=multi threade
As noted above, SDL2 really needs merging of dirty rects as much as possible. THe above number are with the optimized algorithm of getting as few as possible dirty rects.
I like to look at great maps and see how things flow. A little like a finished model railway, but it is evolving and actually never finished. http://www.simutrans.com
Re: Port SDL 1.2 code to SDL 2.0
Are you sure it isn't a vsync issue?
When reading the code,
I read that
in your SDL1 code: async_blit = parameter[0];
SDL2 code: sync_blit = parameter[0];
Perhaps if you try sync_blit = !parameter [0] you'll get better results...
Even if it's not that, the use of SDL of OpenTTD looks very different to yours, so I won't expect similar results.
When reading the code,
I read that
in your SDL1 code: async_blit = parameter[0];
SDL2 code: sync_blit = parameter[0];
Perhaps if you try sync_blit = !parameter [0] you'll get better results...
Even if it's not that, the use of SDL of OpenTTD looks very different to yours, so I won't expect similar results.
Re: Port SDL 1.2 code to SDL 2.0
While the code is different, the general action first rendering to buffer and then copy the dirty areas only to screen are similar. OpenTTD has the added difficulty of a more fixed frame rate, i.e. could not run with 10 fps and similar timing compared to simutrans (or that was my knowledge last time I deeply looked at the OpenTTD code).
I will check for the vsync issues but this was not the only bottleneck. I saw that OpenTTD does not make any attempt to merge dirty rects, which is the key with SDL2. Otherwise SDL2 was even slower (with about 50% more dirty rect covering the same area, SDL2 needed about twice the time for the copy per frame). Copying the whole frame each update was also tried, but with resolutions of fullHD and even more this was very slow. Considering that mobile phones already have such resolutions, together with CPU power (and memory to screen bandwidth) much less than a desktop this becomes even more important.
Finally threading for copying was even slower with SDL2. Speculation was that the hardware blitter could not cope well and one had a queue which was just filled with additional threading overhead.
While SDL2 hardware support is certainly nice to have, oldschool games which did their rendering before in anything other than 32 bit seems a little neglected. Otherwise I see no reason which a portable library is not improving on the platform at least 80% of user are using ...
BTW: I still need to check this, but if SDL2 does not have Unicode composed charecters, then any asian splayer will not be happy with SDL2 ...
I will check for the vsync issues but this was not the only bottleneck. I saw that OpenTTD does not make any attempt to merge dirty rects, which is the key with SDL2. Otherwise SDL2 was even slower (with about 50% more dirty rect covering the same area, SDL2 needed about twice the time for the copy per frame). Copying the whole frame each update was also tried, but with resolutions of fullHD and even more this was very slow. Considering that mobile phones already have such resolutions, together with CPU power (and memory to screen bandwidth) much less than a desktop this becomes even more important.
Finally threading for copying was even slower with SDL2. Speculation was that the hardware blitter could not cope well and one had a queue which was just filled with additional threading overhead.
While SDL2 hardware support is certainly nice to have, oldschool games which did their rendering before in anything other than 32 bit seems a little neglected. Otherwise I see no reason which a portable library is not improving on the platform at least 80% of user are using ...
BTW: I still need to check this, but if SDL2 does not have Unicode composed charecters, then any asian splayer will not be happy with SDL2 ...
I like to look at great maps and see how things flow. A little like a finished model railway, but it is evolving and actually never finished. http://www.simutrans.com
Re: Port SDL 1.2 code to SDL 2.0
I've looked at SDL2 and SDL1.2 code to see if SDL_UpdateWindowSurfaceRects would behave differently (performance-wise) than SDL_UpdateRects for Windows.
It appears that for SDL 1.2, it used to call BitBlt for every damaged rect, while now it calls BitBlt one time, but on the whole screen.
I don't know Windows internal, so I can't say if for OpenTTD it would be a performance benefit or not. I just imagine that this change has been done for a good reason.
prissi: Just out of curiosity, does your fps changes with the change I mentionned? (Was Vsync the problem?) I don't know from where comes your performance loss. But I doubt OpenTTD would be affected, since it doesn't use hardware acceleration to render.
It appears that for SDL 1.2, it used to call BitBlt for every damaged rect, while now it calls BitBlt one time, but on the whole screen.
I don't know Windows internal, so I can't say if for OpenTTD it would be a performance benefit or not. I just imagine that this change has been done for a good reason.
prissi: Just out of curiosity, does your fps changes with the change I mentionned? (Was Vsync the problem?) I don't know from where comes your performance loss. But I doubt OpenTTD would be affected, since it doesn't use hardware acceleration to render.
Re: Port SDL 1.2 code to SDL 2.0
Simutrans does not use hardware acceleration either. But it seems that SDL can use OpenGL hardware acceleration directly (and hence the one big copy).
In the profiling thread in the forum SDL2 got slower and slower the higher the number of dirty rectangles to update. However, full update was even slower. So it may be something like deferred copy to screen and blocking all calls until the last has not been processed or so. Also locking/not-locking the surface was very important.
All things point towards a clash with the new architecture rather than simply some wrong settings in the initialisation. Nevertheless, at OpenTTD 8 or 32 bit depth the situation might be better. (But then OpennTTD already uses DirectX, if memory serves me right.)
In the profiling thread in the forum SDL2 got slower and slower the higher the number of dirty rectangles to update. However, full update was even slower. So it may be something like deferred copy to screen and blocking all calls until the last has not been processed or so. Also locking/not-locking the surface was very important.
All things point towards a clash with the new architecture rather than simply some wrong settings in the initialisation. Nevertheless, at OpenTTD 8 or 32 bit depth the situation might be better. (But then OpennTTD already uses DirectX, if memory serves me right.)
I like to look at great maps and see how things flow. A little like a finished model railway, but it is evolving and actually never finished. http://www.simutrans.com
Re: Port SDL 1.2 code to SDL 2.0
OpenTTD doesn't use SDL for windows (by default, can be compiled in afair), but i think it uses GDI, not DirectXprissi wrote:(But then OpennTTD already uses DirectX, if memory serves me right.)
Re: Port SDL 1.2 code to SDL 2.0
I've checked again the source code, and figured out that :Rubidium wrote:Looking at the migration page and certain APIs I think it is significantly more than just a few #ifdefs.
SDL_SetColors used to get a SDL_Surface, but now SDL_SetPaletteColors needs a SDL_Palette.
SDL_BlitSurface used to get two SDL_Surfaces and two SDL_Rects, but now that needs to become SDL_RenderCopy with a SDL_Renderer, SDL_Texture and two SDL_Rects.
SDL_GetKeyState used to return KeySyms, but now SDL_GetKeyboardState returns ScanCodes.
I'd reckon that creating a new SDL2 video driver will result in easier to read code, and less problems overall. It will also allow one to make more use of the newer features of SDL2.
. SDL_UpdateWindowSurfaceRects is equivalent to SDL_UpdateRects as I said.
. Some platforms specify a particular optimized behaviour for it (as for SDL_UpdateRects)
. But as I said in some situations it will prefer using GL textures, and use GL to commit the window content.
In this situation, it'll do exactly the same thing that what is describe in the SDL1.2 -> SDL2 tutorial (part "If your game just wants to get fully-rendered frames to the screen").
. SDL_BlitSurface does still exist.
. SDL_GetKeyFromScancode can convert a scancode given by SDL_GetKeyboardState to a SDL_Keycode. SDL_GetKeyName can get a utf-8 character from it. If it helps replace SDL_GetKeyState...
Re: Port SDL 1.2 code to SDL 2.0
Is anybody still considering adding support for SDL 2, either by porting the SDL 1 code, or by adding a separate video driver?
Currently there is no support for pasting from the clipboard when compiling on UNIX systems, which means that long names such as yapf.rail_firstred_twoway_eol have to be typed at the console. SDL 2 does support pasting, and on the other hand adding support for X11 clipboard bypassing SDL would be a huge mess.
Currently there is no support for pasting from the clipboard when compiling on UNIX systems, which means that long names such as yapf.rail_firstred_twoway_eol have to be typed at the console. SDL 2 does support pasting, and on the other hand adding support for X11 clipboard bypassing SDL would be a huge mess.
Re: Port SDL 1.2 code to SDL 2.0
Is 3 years of silence not a sufficient answer to your question?lcd_47 wrote:Is anybody still considering adding support for SDL 2, either by porting the SDL 1 code, or by adding a separate video driver?
Being a retired OpenTTD developer does not mean I know what I am doing.
Re: Port SDL 1.2 code to SDL 2.0
Is my question not enough of a hint that I'm actually trying to politely suggest changing the answer?Alberth wrote:Is 3 years of silence not a sufficient answer to your question?
Re: Port SDL 1.2 code to SDL 2.0
well, nobody prevents you from changing the answer by working on it...
Re: Port SDL 1.2 code to SDL 2.0
Not my itch, sorry. My itch (that I might scratch at some point) is that copying from clipboard doesn't work on UNIX. The "huge mess" solution that bypasses SDL would be good enough for solving that.Eddi wrote:well, nobody prevents you from changing the answer by working on it...
Re: Port SDL 1.2 code to SDL 2.0
Fix that, and as a side effect, port to SDL2 
Being a retired OpenTTD developer does not mean I know what I am doing.
Re: Port SDL 1.2 code to SDL 2.0
Hey, so I've been working on this over the past few days. I've gotten OpenTTD's SDL driver working on SDL 2 on this branch: https://github.com/nikolas/OpenTTD/tree/sdl2 I'm probably going to clean this up, fix up loose ends and make a pull request at some point.
Re: Port SDL 1.2 code to SDL 2.0
There was some discussion about dirty rectangles in this thread. Here's a relevant part from the SDL2 MigrationGuide: https://wiki.libsdl.org/MigrationGuide
The dirty rect system is already in place in OpenTTD's SDL video driver, so it couldn't hurt to use it. But when I updated parts of the screen in a for loop like the 1.2 driver does, I saw some intense flickering. Though that may be avoided somehow. But I haven't seen any problems with this entire screen update, and I'm all for simple solutions that just work.
Also, as I've noted in the pull request (https://github.com/OpenTTD/OpenTTD/pull/7086), I'm specifying the SDL 2 renderer as a software renderer for stability reasons. Messing with the GPU sounds nice but introduces some problems that may not work on everyone's setups.
So.. that's exactly what I've done in my SDL2 pull request, here: https://github.com/OpenTTD/OpenTTD/pull ... 323763L158At the end of the frame, we want to upload to the texture like this:
SDL_UpdateTexture(sdlTexture, NULL, myPixels, 640 * sizeof (Uint32));
This will upload your pixels to GPU memory. That NULL can be a subregion if you want to mess around with dirty rectangles, but chances are modern hardware can just swallow the whole framebuffer without much trouble.
The dirty rect system is already in place in OpenTTD's SDL video driver, so it couldn't hurt to use it. But when I updated parts of the screen in a for loop like the 1.2 driver does, I saw some intense flickering. Though that may be avoided somehow. But I haven't seen any problems with this entire screen update, and I'm all for simple solutions that just work.
Also, as I've noted in the pull request (https://github.com/OpenTTD/OpenTTD/pull/7086), I'm specifying the SDL 2 renderer as a software renderer for stability reasons. Messing with the GPU sounds nice but introduces some problems that may not work on everyone's setups.
Who is online
Users browsing this forum: No registered users and 20 guests