suggestion regaring more .grf's allowed at once

[Zephyris]

We need a developer! I'm not experienced enough in c to do it either

[MagicBuzz]

I started to look the code this morning.

Currently, the engines are stored in a 256 length array.

Different types of vehicules uses different ranges of IDs according this description :

Code: Select all

enum {
	NUM_NORMAL_RAIL_ENGINES = 54,
	NUM_MONORAIL_ENGINES    = 30,
	NUM_MAGLEV_ENGINES      = 32,
	NUM_TRAIN_ENGINES       = NUM_NORMAL_RAIL_ENGINES + NUM_MONORAIL_ENGINES + NUM_MAGLEV_ENGINES,
	NUM_ROAD_ENGINES        = 88,
	NUM_SHIP_ENGINES        = 11,
	NUM_AIRCRAFT_ENGINES    = 41,
	TOTAL_NUM_ENGINES       = NUM_TRAIN_ENGINES + NUM_ROAD_ENGINES + NUM_SHIP_ENGINES + NUM_AIRCRAFT_ENGINES,
	AIRCRAFT_ENGINES_INDEX  = NUM_TRAIN_ENGINES + NUM_ROAD_ENGINES + NUM_SHIP_ENGINES,
	SHIP_ENGINES_INDEX      = NUM_TRAIN_ENGINES + NUM_ROAD_ENGINES,
	ROAD_ENGINES_INDEX      = NUM_TRAIN_ENGINES,
};

For backward compatibility, we can't change the way the first 256 vehicule IDs are stored.

So, we need to store the extended IDs after these lines.
I got 2 ideas :

0x00xx = standard vehicules
0x1xxx = extended vehicules type 1
0x2xxx = extended vehicules type 2
etc. until the 6th vehicule type

Using this, we can store up to 4096 new vehicules of each 6 types, and we keed enought space to extend in with 10 new vehicules types (could be usefull for subways, new rails types trains, etc.)

But the big problem come with the enumeration of the vehicules of a certain type : it's hard to get the range of vehicules from the standard vehicules IDs range, plus the new range.


The second solution I found is to reproduce the same structure for extended vehicules IDs.

0x01xx = 2nd set of 256 "standard set".
0x02xx = 3rd
...
0xFExx = last

(we can't use the 0xFFxx range as the 0xFFFF is already used for the "INVALID_ENGINE_ID" constant)

By this way, we can find easily all vehicules of a certain type. But I don't like the way it works, as we can't add new vehicule types later.

For both cases, I started to implement theim and rollbacked all my changes as I can't understand all parts of code I modify

[Zephyris]

I think the second option for two reasons (despite it being less tidy!)

* if any new rail types etc. are added, it is likely that itll be done via ttdpatch, and so slot into the existing "classic" organisation.
*it would allow much simpler implementaion of the loading of more than one "old style" B vehID sets simeltaneously (eg. UKRS and NARS together )

[DaleStan]

Is there any reason for the massive bias of grf development toward ttdpatch?

Yes.
Patch had newgrf support long before OpenTTD existed, and is still the only complete implementation. Open keeps working on improving its support, and I expect it will have a complete implementation eventually, but that's probably at least a year in the future. More if either newroutes or newobjects get completed in that time.

[MagicBuzz]

Good news !

I didn't changed anything in the newgrf implementation, but I changed all the code about the current 256 vehicules limitation in OTTD.
I duplicated all the basic vehicules with higher IDs (standard ID + 256) and I'm able to see them in the "build new vehicle" window.

Teaser : all standard vehicles duplicated. I used a small GRF file with a standard vehicule ID to replace a regular vehicule (ID 22). As we can see, the "plop" vehicule is displayed with the patched graphics (see the red dots) while the original with a 278 (22 + 256) is still displayed. In the picture I replaced the vehicule's weight with it's ID, so we can see the standard unmodified vehicule is ID 278

extended_engine_id_1.png (7 KiB) Viewed 2617 times

Teaser : all standard vehicles duplicated. I used a small GRF file with a standard vehicule ID to replace a regular vehicule (ID 22). As we can see, the "plop" vehicule is displayed with the patched graphics (see the red dots) while the original with a 278 (22 + 256) is still displayed. In the picture I replaced the vehicule's weight with it's ID, so we can see the standard unmodified vehicule is ID 278


Now, I have to update all the newgrf code to handle this also in order to build a ID >= 256 vehicule.

I hope I'll be able to produce an Alpha version tomorrow (I'm not sure I can finish this evening... :/)

[Zephyris]

Some basic GRF .nfo coding to help you test GRF support.
## indicates the point where the VehicleID belongs
Sorry if this is patronisingly simple, I don't know how much grf experience you have!

Action0

Code: Select all

// Automatically generated by GRFCODEC. Do not modify!
// (Info version 7)
// Format: spritenum pcxfile xpos ypos compression ysize xsize xrel yrel
    0 * 4	 02 00 00 00
    1 * 34	 08 06 "B*" 01 00 "B* VehID Test GRF (Action0)" 00
    2 * 7 	00 //Action0
		00 //Feature00 (Trains)
		01 //1 property changed
		01 //For 1 vehicle
		## //VehID of altered vehicle
		00 //Property00 (Date of introduction)
		00 00 //Year=1920

Action4

Code: Select all

// Automatically generated by GRFCODEC. Do not modify!
// (Info version 7)
// Format: spritenum pcxfile xpos ypos compression ysize xsize xrel yrel
    0 * 4	 02 00 00 00
    1 * 34	 08 06 "B*" 01 04 "B* VehID Test GRF (Action4)" 00
    2 * 18 	04 //Action0
		00 //Feature00 (Trains)
		0F //Language (All)
		01 //For 1 vehicle
		## //VehID of altered vehicle
		"Action4 Test" //Name string
		00 //String termination

Action2 (+1 and 3)

Code: Select all

// Automatically generated by GRFCODEC. Do not modify!
// (Info version 7)
// Format: spritenum pcxfile xpos ypos compression ysize xsize xrel yrel
    0 * 4	 02 00 00 00
    1 * 34	 08 06 "B*" 01 03 "B* VehID Test GRF (Action3)" 00
    2 * 4	 01 00 01 08
    3 c:\games\openttd\sprites/VehIDTestGRF.pcx 98 8 01 18 8 -3 -10
    4 c:\games\openttd\sprites/VehIDTestGRF.pcx 114 8 09 16 20 -14 -7
    5 c:\games\openttd\sprites/VehIDTestGRF.pcx 146 8 01 12 28 -14 -6
    6 c:\games\openttd\sprites/VehIDTestGRF.pcx 194 8 09 16 20 -6 -8
    7 c:\games\openttd\sprites/VehIDTestGRF.pcx 226 8 01 18 8 -3 -10
    8 c:\games\openttd\sprites/VehIDTestGRF.pcx 242 8 09 16 20 -14 -8
    9 c:\games\openttd\sprites/VehIDTestGRF.pcx 274 8 01 12 28 -14 -6
   10 c:\games\openttd\sprites/VehIDTestGRF.pcx 322 8 09 16 20 -6 -7
   11 * 9 	 02 00 AA 01 01 00 00 00 00
   12 * 7 	03 //Action3
		00 //Feature (trains)
		01 //1 vehicle modified
		## //VehID
		00 //No cargo specific graphics
		AA 00 //Default action2ID

[MagicBuzz]

Faster than expected...

Here is a teaser :

extended_engine_id_2.png (34.13 KiB) Viewed 2586 times

As you see I was able to build both standard and altered vehicules and I can use them.

But it's far to be finished : newgrf system must be reviewed to be able to read extended IDs and to use them.

Right know, I just found something I'm sure I can't change by myself...

Code: Select all

void AlterRailVehListOrder(EngineID engine, EngineID target)
{
	/* MagicBuzz : wow... will be very hard to change !!! */

	EngineID i;
	bool moving = false;

	if (engine == target) return;

	/* First, remove our ID from the list. */
	for (i = 0; i < NUM_TRAIN_ENGINES - 1; i++) {
		if (_engine_list_order[i] == engine) moving = true;
		if (moving) _engine_list_order[i] = _engine_list_order[i + 1];
	}

	/* Now, insert it again, before the target engine. */
	for (i = NUM_TRAIN_ENGINES - 1; i > 0; i--) {
		_engine_list_order[i] = _engine_list_order[i - 1];
		if (_engine_list_order[i] == target) {
			_engine_list_order[i - 1] = engine;
			break;
		}
	}

	/* Update the engine list position (a reverse of engine list order) */
	for (i = 0; i < NUM_TRAIN_ENGINES; i++) {
		_engine_list_position[_engine_list_order[i]] = i;
	}
}

It seems this funcion is used to alter the vehicules order according the GRF file. As now the vehicules of the same type are not stored together, it means we have to update vehicule order for all the ranges between the 2 moved vehicules.

You can find my current patch here (pretty unusefull, but any help/review/debug is welcome).

To change the number of available sets, you can change the "MAX_ENGINE_SET" value in engine.h, then duplicate all standard vehicules definition in the engines.h (take a look at my patch to see what I mean).
To hide the duplicated vehicules, just try to remove any climate flag in the "orig_engine_info" like i did for the duplicated "Kirby Paul Tank (Steam)" (line 324)


PS : This bumps the save format (it seems it stores all available IDs, as I didn't do anything for), so don't be chocked in the opening screen shows a big sea instead of the usual scenery.

[XeryusTC]

I wonder why you want to change the NFO spec, while you could better assign IDs automatically, use the normal IDs for the default vehicles, and start numbering from 256 for every loaded vehicle set.

[MagicBuzz]

I wonder why you want to change the NFO spec, while you could better assign IDs automatically, use the normal IDs for the default vehicles, and start numbering from 256 for every loaded vehicle set.

Because my background idea is not to allow multiple sets to be used at the same time, but using sets without the current limitation of vehicules numbers.

This might allow more wagon types by exemple, as currently, if you design low speed wagons and high speed wagons, you won't be able to design middle speed wagons without compromising the number of engines.
Allowing more vehicules ID may allow us to get different wagons for different rails types also.

This could also allow to get in one GRF complete sets for multiples climates, etc.

In my tests, I duplicate the basic vehicules, but you have to keep in mind all "standard vehicules" with ID > 255 won't be visible in the release version. Only GRF will add usable vehicules.

[DaleStan]

Allowing more vehicules ID may allow us to get different wagons for different rails types also.

May allow!? I assume you don't mean different wagons for standard and electrified rail here? In all other cases, it's already required that wagons for different rail-types require different IDs.

This could also allow to get in one GRF complete sets for multiples climates, etc.

You obviously haven't inspected the USSet lately. Which engine uses a given ID is climate-dependent. This is probably true of other sets too.

[Zephyris]

As an intermediate step a patch which allows loading of multiple vehicle newgrfs into separate slots may be more appropriate, I for one would love to take advantage of this. In particular I would like to:
* Have more than 40 aeroplanes and more than 10 ships (by cooperation of two files)
* Have a better vehicle spread across time (1920 to past 2050) (by cooperation of two files)
* "Modular" newindustries (when supported); each grf would carry the industries along with neccecary wagons, road vehicles, and boats with no fear of VehID clashes.

[MagicBuzz]

May allow!? I assume you don't mean different wagons for standard and electrified rail here? In all other cases, it's already required that wagons for different rail-types require different IDs.

Yeah, but what about doing 3 sets of wagons for each rail type ?

You obviously haven't inspected the USSet lately. Which engine uses a given ID is climate-dependent. This is probably true of other sets too.

A vehicule just contains information telling "this vehicule is visible in these climates".
So what about using 50 engines in temperate only, 50 for tropical only, 50 for artic only, and 50 shared between temp and trop, 50 shared between temp and arct, plus 50 shared with all climates ? What about using different wagons for each climates ? Algain you'll consume a lot of ID is you design 2 sets of wagons for each climates.

Currently everyone just say "it's not usefull, 256 Ids are enougth", but as I see :
1/ Most of big sets use all IDs : allowing more will for sure allow the author to add some. I think about cargo / long range passengers / suburban passengers liverys. Most of the companies in the world uses same engines for different services, but with different liveries. Adding IDs will just help the authors to produce alternatives liveries, that are totally optional.
2/ Several people are complaining about the limitation of planes and boats range IDs that are too small. This feature just allow to get ride of this limitation.

[MagicBuzz]

As an intermediate step a patch which allows loading of multiple vehicle newgrfs into separate slots may be more appropriate, I for one would love to take advantage of this. In particular I would like to:
* Have more than 40 aeroplanes and more than 10 ships (by cooperation of two files)
* Have a better vehicle spread across time (1920 to past 2050) (by cooperation of two files)
* "Modular" newindustries (when supported); each grf would carry the industries along with neccecary wagons, road vehicles, and boats with no fear of VehID clashes.

I won't design a system that change the GRF vehicule ID. If you want to combine 2 GRFs, just decode the second one, change the vehicule IDs, and compile it again, it will work (when the GRF support will be patched).
The goal of my patch is not to combine multiple sets, but get ride of the limitation of vehicule ID. If one set is designed to add vehicules to a certain set, it will work as you want to, just by adding higher vehicules ID.

For new industry support, I think the main problem comes with the CargoID. Don't know how it works now, and how it should with new industries support. If we can easily increase it, then it will be automatically supported by this patch. We just have to ensure the new industry GRF will add new Cargo IDs and not replace olders.

[Zephyris]

My point is that once the game can handle extended byte vehicle IDs then there are two ways this can be used; updating the grf specs and/or allow loading of multiple newgrfs into separate "slots". Whilst the first is preferable the second is easier to implement (no change to newgrf specs) and easier to use with current sets (no decoding and editing required). This also makes it a good option to be added to trunk - it provides an instant improvement and is wholy compatible with existing newgrfs. In the future (the newgrf spec may well take months to change) the this could be further updated.

[DaleStan]

You obviously haven't inspected the USSet lately. Which engine uses a given ID is climate-dependent. This is probably true of other sets too.

A vehicule just contains information telling "this vehicule is visible in these climates".

Yeah? So? Why would you have a GRF load a vehicle in a climate where the vehicle can't be used?

And please learn how to spell "vehicle". It does not contain a ewe.

[MagicBuzz]

You obviously haven't inspected the USSet lately. Which engine uses a given ID is climate-dependent. This is probably true of other sets too.

A vehicule just contains information telling "this vehicule is visible in these climates".

Yeah? So? Why would you have a GRF load a vehicle in a climate where the vehicle can't be used?

And please learn how to spell "vehicle". It does not contain a ewe.

I mean if I design an "america" train set, I could get 3 climates :
- USA = temperate
- Canada = arctic
- Latina america = tropical

Canada and latina america use for sure USA vehicles, but with different introduction dates, specs and reliability : an engine designed for temperate condition can become awfull with tropic or arctic climates.

Allowing the graphist to use more Engine IDs will help him distributing his creation, as he can distribute it as a unique file, while it covers any available slots for all climates.

PS : I'm sorry for the vehicle spelling, in French it's vehicule so I make confusion and sometimes I use the French spelling

[MagicBuzz]

My point is that once the game can handle extended byte vehicle IDs then there are two ways this can be used; updating the grf specs and/or allow loading of multiple newgrfs into separate "slots". Whilst the first is preferable the second is easier to implement (no change to newgrf specs) and easier to use with current sets (no decoding and editing required). This also makes it a good option to be added to trunk - it provides an instant improvement and is wholy compatible with existing newgrfs. In the future (the newgrf spec may well take months to change) the this could be further updated.

Ok, I'll try to see that, but I'm pretty sure it's almost impossible : GRFs actually don't create vehicules, but modify (or override) the standard ones.
Because of this, if I apply an offset when loading, I can't actually know if the standard vehicle was already modified or not.
Even if I can detect it, anu GRF "fix" won't work anymore (I think about the UKRS update) : the previous modified vehicle will remain like this, while the fix will be applied to an unexpected vehicle.

Exemple :
"first.grf" will upade the vehicle ID 0 with a complete set of new specs. As the graphist is a dumb French like me, he named the vehicle "vehicule 1"
After a few months, he decide to publish a fix in order to fix the spelling only for english language (he didn't make a mistake for the other languaues).
So he publishes a new grf "second.grf" with only one action : rename vehicle ID 0 in english to "vehicle 1".

If OTTD applyes an offset, then this patch will rename the vehicle 256. 256 is the Kirby Paul Tank (if we simply duplicated the standard vehicles), not at all the vehicle defined in the "first.grf".

Just expect many GRF won't work as expected if we just apply a offset on the ID.

Also, the big problem is that all engines will take memory. If we decide to duplicate the vehicle for each read GRF file, we will consume a lot of memory.

A simple way to fix it is to add an option for the GRF files : "offset=X". If omitted, then the IDs will be loaded as 0-255 range. If filled, then the IDs will be changed to "read ID + 256 * <option value>".
This might work and allow the player to chosse precisely which vehicles will be stored with extended IDs.

(but I don't have ANY idea on how to implement it)

Here is my lastest patch (with some minor changes in the newgrf.cpp file)

[DJ Nekkid]

any news?

[DaleStan]

I suspect that if there were, it would have been posted.

[Aydan]

How about extending the NewGrf Dialog to ba able to assign a Slot(0-255) to each NewGrf? This way you could still have the ability to update other grfs combined with the ability to use normally overlapping grs at the same time.