Page 1 of 1

Merging Yacd, Nogo

Posted: 16 Dec 2011 17:24
by Aphid
I want to apply the two patches attached to this post to an openTTD.

Unfortunately, after spending an entire day installing various tools and confusing with /, it seems I am no nearer as I can only get myself what seems to be a previous version of the basic game with no sound or music. (What with MS removing directmusic support ages ago and the library provided by you guys being 32bit? how do I link it?)

One patch is a git patch, the other a svn. And again, the two seem to hate eachother. (Also, they're made against varying versions, but that shouldn't surprise anyone.).

So, my question is, how and what do I need to do, exactly, in minute detail, everything attached, given the following config;
Win 7 x64, VS2010, KDiff3, msysgit, msys, TortoiseSVN & Git, the bunch of stuff in the dependancy pack, and (likely) some other stuff I forgot.

Or, alternately, could someone build it for me?

(It would be nice if there'd be a yacd for 1.1.4 though, something to forward)


-- Note; it is to combine yacd with the now popular 'citybuilder' concept. This might yield interesting results, as I find city development better with cb.

Re: I give up ~ Patches

Posted: 16 Dec 2011 17:31
by Rubidium
There is no directmusic for 64 bits binaries, so no need to link it.

Furthermore, the source versions of those patches are so far apart that they won't apply on the same codebase. So you need to update (or downgrade) on of the patches to the source version of the other patch. Otherwise all attempts of merging the patches will go horribly wrong.

After that it is 'simply' applying both patches and fixing all the rejects of the patching process which means you really need to know what you're doing and writing that down in concise wording is impossible. You simply need to know how to program in C++, and how OpenTTD internally works to be able to solve the rejects. Although you could try the "monkey" method and let enough monkeys loose on the two patches until they are merged.

Re: I give up ~ Patches

Posted: 16 Dec 2011 18:32
by Aphid
Ah... fortunately, I do understand C++, so that's nice, but a bit of work for me then :arrow: .
I just had a bit of trouble dealing with all these new programs, but at least got a basic version to compile. So all that is left is to get it patched.

It would seem that the best course of action would be to get 1.1.4 source, patch on citybuilder, and then just read through the .diff file of the incompatipatch (YACD), applying the changes manually?
Indeterminate workload being the only concern here. 1.1.4 is the later version, so there should be less compatibility issues then, logically, or is that illogical? 'Puts on spock voice here

EDIT: It was easy to get sound working, actually...
Edit no. 2; patching the first patch went okay as well, so now the hard part remains;

I wonder if it's possible to preserve what needs to be done so that it's only done once.


Note: Please edit the OTTDpedia article about compiling and needing to install old DirectX SDK's and all that.
Having a relatively new DX11 SDK, and simply downloading both source and the dependancy pack is enough to get it working. (I can't ever remember installing an SDK older than VS2008)
Include directories also did not need to be changed in the VS10 project. To be fair, a page did not exist for it. I can see why now, but I can also see how I got confused here~

Re: I give up ~ Patches

Posted: 16 Dec 2011 18:53
by Alberth
Patching a stable release is just nonsense. After you patched it, it is not compatible with the stable version anymore. In other words, there is no useful extra value for patching a stable release, you can just as well pick any trunk revision.

So you better use a trunk version imho, and start from there.


(No, 1.1.4 is not the newer version. It got branched from trunk last januari, and got only backported patches. Trunk on the other hand is on its way to 1.2, and contains new features.)


And about wiki editing: It's a wiki, you know? :p
You are in the best situation to fix it, as you seem to have knowledge that we don't have.

Re: I give up ~ Patches

Posted: 16 Dec 2011 19:00
by ChillCore
/me pulls the emergency brake ...

Forget about patching 1.1.4 and patch a trunk revision.
1.1.4 is based on the same revision as 1.1.0 as you can see here http://wiki.openttd.org/FAQ_OpenTTD_versions#Trunk
Everything on the 1.1.x branch is backports after 1.1.0 was branched.

Your method of patching one pach to the correct revision and then applying the second one manually will work if you want to go that way ... I do that too most of the times. Do keep in mind that you may have to change the syntax of the second patch though.

Just so you know: You can bump revisions while a patch is applied too ... you may have to fix a few conflicts but this is a lot less work then applying patches manually to a later revision (-> depends heavily on the used patches and trunk changes in between revisions).

So if I were you ... apply the patch based on the lowest revision first to its correct revision, bump to the revision of the second patch (while fixing conflicts) then apply the other patch (while fixing conflicts). The more patches you have applied at the same time the more you will have to fix but I guess you are aware that. ;)




Edit:
Alberth was faster once again ... You stalking me Alberth? :mrgreen:

Re: I give up ~ Patches

Posted: 16 Dec 2011 19:07
by planetmaker
It might be much more interesting to patch YACD into the source of the NoGo branch.

For that endeavour you could pull directly from the git (yacd) and hg (nogo) repositories respectively. That shouldn't be more difficult than using patches based on potentially older versions of those two things.

Re: I give up ~ Patches

Posted: 16 Dec 2011 20:19
by Aphid
Okay, I'll go ahead and start doing that then. I can see why, this NoGo has more possibilities.

(this thread is no longer in the right forum. It will probably be in the future though)

Re: I give up ~ Patches

Posted: 16 Dec 2011 22:57
by Yexo
Development problems are fine in the development section, so I've moved it over there.

Re: I give up ~ Patches

Posted: 26 Jan 2012 10:57
by Aphid
I'm progressing quite nicely in what I want to do (done merging the two branches, but now to get the gameplay balanced and more of such things), but I've got another question. Something which quite stumped me was that there seems to be little information on the workings of strgen, at least that I can find.

From what I was able to figure out, some change I did somewhere else made something go wrong in strgen. That's quite vague, but it simply exits with code 1, I can't seem to get to any other output of this program. (When run in a command window, with any parameter, even -h, it opens a new window, closes that five milliseconds later, and is done running).

Okay, no problem, maybe I need to delete the auto-generated string.h file. Or change file permissions. Both have no effect. Perhaps strgen.exe needs admin privileges (since it changes files), no effect. Well, I'll just see if I can insert breakpoints in strgen's main() function at the points where it returns 1. Find out where it errors. So the funny thing is, inserting a breakpoint anywhere does not seem to halt strgen. The important part of the specific compiler error message is;

Code: Select all

error MSB3073: The command "..objsstrgenstrgen.exe -s ..srclang -d ..objslangstable
error MSB3073: :VCEnd" exited with code 1.
Are there any other ways this error message could come up other than what it actually indicates? Searching on the internet got me a lot of stuff that actually wasn't related and a multitude of diffrent answers, so this might be it. (microsoft error messages can be like that)

edit: Added proper backslashes.

edit: the above problem has been fixed.

Re: I give up ~ Patches

Posted: 26 Jan 2012 11:25
by Alberth
strgen shares a bit of code with the main openttd program, so perhaps you did make changes in it without being aware of it.
(look at which source files it uses to build strgen)

Other than that, your error looks like a Windows error where it fails to start the program (due to some missing library or dll, or whatever you have at Windows, for example).
That would also explain why -h does not do anything.

Re: I give up ~ Patches

Posted: 26 Jan 2012 13:01
by Aphid
There's been a change in the YAPF recently, it seems that feeding a NULL vehicletype into it no longer translates into it trying combinations of vehicletypes. Instead it's always returning NULL paths. (Which leads to the interesting behaviour of no cargo appearing at all).

There's probably been a check added for it of some sort somewhere. But since it feeds the vehicletype pointer into a class member called m_veh and that occurs all of five million times in the code, it's kind of a needle in a haystack. Anyone know where I'd find this?

Or should we rather try pathfinding combinations of vehicles...

Also, some experimentation lead to the conclusion that industries seem to view the world in a more pessimistic way than towns. Towns still seem to believe their people are being transported even as they're sucked into the void by the pathfinder. Industries do not. What that actually means I don't know yet. It's probably a good clue though.

Re: Merging Yacd, Nogo

Posted: 27 Jan 2012 18:46
by Aphid
Well, now I'm completely stumped.
After rummaging around (a lot) in the pathfinder, here's the weirdest bug I've ever seen yet.

Code: Select all

/**
	 * AddNewNode() - called by Tderived::PfFollowNode() for each child node.
	 *  Nodes are evaluated here and added into open list
	 */
	void AddNewNode(Node &n, const TrackFollower &tf)
<A>	{
		/* evaluate the node */
		bool bCached = Yapf().PfNodeCacheFetch(n);
<B>		if (!bCached) {
			m_stats_cost_calcs++;
		} else {
			m_stats_cache_hits++;
		}

		bool bValid = Yapf().PfCalcCost(n, &tf);

		if (bCached) {
			Yapf().PfNodeCacheFlush(n);
		}

		if (bValid) bValid = Yapf().PfCalcEstimate(n);

		/* have the cost or estimate callbacks marked this node as invalid? */
		if (!bValid) return;

		/* detect the destination */
		bool bDestination = Yapf().PfDetectDestination(n);
		if (bDestination) {
			if (m_pBestDestNode == NULL || n < *m_pBestDestNode) {
				m_pBestDestNode = &n;
			}
			m_nodes.FoundBestNode(n);
			return;
		}

		if (m_max_search_nodes > 0 && (m_pBestIntermediateNode == NULL || (m_pBestIntermediateNode->GetCostEstimate() - m_pBestIntermediateNode->GetCost()) > (n.GetCostEstimate() - n.GetCost()))) {
			m_pBestIntermediateNode = &n;
		}

		/* check new node against open list */
		Node *openNode = m_nodes.FindOpenNode(n.GetKey());
		if (openNode != NULL) {
			/* another node exists with the same key in the open list
			 * is it better than new one? */
			if (n.GetCostEstimate() < openNode->GetCostEstimate()) {
				/* update the old node by value from new one */
				m_nodes.PopOpenNode(n.GetKey());
				*openNode = n;
				/* add the updated old node back to open list */
				m_nodes.InsertOpenNode(*openNode);
			}
			return;
		}

		/* check new node against closed list */
		Node *closedNode = m_nodes.FindClosedNode(n.GetKey());
		if (closedNode != NULL) {
			/* another node exists with the same key in the closed list
			 * is it better than new one? */
			int node_est = n.GetCostEstimate();
			int closed_est = closedNode->GetCostEstimate();
			if (node_est < closed_est) {
				/* If this assert occurs, you have probably problem in
				 * your Tderived::PfCalcCost() or Tderived::PfCalcEstimate().
				 * The problem could be:
				 *  - PfCalcEstimate() gives too large numbers
				 *  - PfCalcCost() gives too small numbers
				 *  - You have used negative cost penalty in some cases (cost bonus) */
				NOT_REACHED();
			}
			return;
		}
		/* the new node is really new
		 * add it to the open list */
		m_nodes.InsertOpenNode(n);
	}
Putting a breakpoint at <A> results in that breakpoint getting hit everytime a new cargopacket is made and FindPath() is called (after a series of calls). Putting a breakpoint at <B> however, causes that breakpoint to never get hit.

Uh, how is that possible? I do not understand. Here's what PfNodeCacheFetch does:

Code: Select all

		
// no cache version
FORCEINLINE bool PfNodeCacheFetch(Node& n)
	{
		return false;
	}
// local cache version
	FORCEINLINE bool PfNodeCacheFetch(Node& n)
	{
		CacheKey key(n.GetKey());
		Yapf().ConnectNodeToCachedData(n, *new (m_local_cache.Append()) CachedData(key));
		return false;
	}
So what exactly is causing this behaviour?

Re: Merging Yacd, Nogo

Posted: 27 Jan 2012 19:11
by Alberth
First thought is that you did not disable code optimization sufficiently.

By default, C++ compilers move and merge code a lot to reduce CPU costs.

Unless you disable that during compilation, there is a chance the code that you see/select doesn't exist any more in the sense that it is merged with other code, or it is rerouted to another place.

Re: Merging Yacd, Nogo

Posted: 27 Jan 2012 19:15
by Eddi
to me it looks like the function always returns false. so quite possible that the compiler (or linker) optimized that away completely

Re: Merging Yacd, Nogo

Posted: 27 Jan 2012 19:20
by Aphid
That still doesn't solve the issue but at least I found why my search was leading nowhere. So turned optimization off. It's still a goose chase trying to find exactly where I'm getting a wrong value.
In that same function (linked up one post of mine), it always returns before adding an open node, if and only if m_veh (somewhere completely diffrent?) is NULL.
As to the specific point where it seems to fly off, it also seems that the code path does not go further than the /* detect the destination */ section. (in turn putting a NULL value into the best node, causing FindRoute to return NULL)

Seems that Yapf().PfDetectDestination(n); always returns true. This should translate to;

n.GetRouteLink() being NULL, which means

n.m_key.m_link; being NULL.

Which means.... what exactly? (aside from the pathfinder always instantly giving up the first algorithm loop for some as-of-yet inexplicable reason.)