Request feature for the GRF Crawler site

Andrex · Post by **Andrex** » 20 Jan 2007 00:25

I wonder if the GRF Crawler admin(s) could create an RSS file containing latest graphics and stuff, in order to allow TTDXC do directly check for latest graphics, versions, and download an install graphics by user request.

By RSS file I mean an RSS 2.0 file such as this, something simple with graphics names, version, date and page URL.

I tried contacting the site admin without success, maybe I didn't send the mail to the proper address, the site does not provide a proper email.

Post by **PikkaBird** » 20 Jan 2007 01:25

But GRFCrawler doesn't know when a grf file is updated - it's just a directory. It also doesn't have direct download links for grf files.

Rob · Post by **Rob** » 20 Jan 2007 04:25

But the graphic crawler has an RRS feed.

DaleStan · Post by **DaleStan** » 20 Jan 2007 05:48

But it still doesn't have direct download links. And it doesn't want them either; the description of the download link explicitly says "Download Website where the grf can be easily found, not a direct link to a file" (Emphasis mine.)

Post by **PikkaBird** » 20 Jan 2007 07:59

Rob wrote:But the graphic crawler has an RRS feed.

It still doesn't know when grf files are updated; it only knows when grfcrawler entries are updated.

eis_os · Post by **eis_os** » 20 Jan 2007 09:33

As grf author denied the distribution of grfs via grfcrawler, there is no way to tell if a grf is updated, what md5sum it has, neither can it store old versions so you can run old savegames without breaking...

The rss feed pulls out the 10 last updates (means new entries aswell) at the database and shows them... The result is, that the there is no real sync between grfs and grfcrawler entries.

-edit-
I forget to upload my last changes about rss,
I hope all RSS feed readers still understand the xml now...

While it still can't produce 100% valid rss2 feeds (The concept of guids don't work with grfcrawler) it gets better in every version...

Andrex · Post by **Andrex** » 20 Jan 2007 19:14

Direct GRF download link is not needed, the graphic's page URL is enough. Having the 10 latest updates is enough as well.

I think the GRF Crawler site is the only proper manner to make graphics massively available to public, so graphic authors should seriously consider releasing their work there as soon as it's ready for publishing it. If an author doesn't want his graphic published there, a link to his site would be enough. Otherwise, they shouldn't bother in making the new graphics at all, if they don't want them to be used by TTD players.

I have one question regarding the RSS feed: Why the GRF description is inside a <content:encoded> tag? Why not use a plain <description> tag, which is widely used on most RSS feeds?

eis_os · Post by **eis_os** » 20 Jan 2007 19:40

Content:encoded provides the full grfcrawler description entry currently.

Specs:
http://web.resource.org/rss/1.0/modules/content/

Wordpress used this way aswell, in some versions, it looks like they now use both ways, s9y still uses Content:encoded.

While I should provide generally all informations at the Content:encoded part, including grfid an stuff, I haven't seen much problems yet. May I ask what feed application you use for testing?

Most times the grfcrawler entry would be to much for description anyway I think...

The specs however may change without notice, as I said, it's still work on progress, so I generally I had the plan to add an additional grfcrawler xml namespace for providing the grfid as example...

Andrex · Post by **Andrex** » 21 Jan 2007 02:29

My original idea was to add a new tab named GRF Crawler and create an HTML page inside it, displaying a list of latest graphics including a small screenshot, to provide the user a preview of each graphic set. The user cliks one and a small broswer window would open directing him to the proper GRF Crawler page.

To do so I'll have to use the current MSXML parser, which TTDXC already uses for getting all TTDPatch related info. But the RSS currently does not include an URL for a screenshot. As soon as this is implemented, I'll be able to start working on this.

I've just finished dealing with the movie trailers RSS on http://www.divxland.org , check any page there (not TTDXC's page) and scroll to the bottom of the page to view the result. Only after that I got the idea to use GRF Crawler's RSS for to make graphics more easily available to TTDXC users.

DaleStan · Post by **DaleStan** » 21 Jan 2007 02:50

Andrex wrote:To do so I'll have to use the current MSXML parser, which TTDXC already uses for getting all TTDPatch related info. But the RSS currently does not include an URL for a screenshot.

The entry with ID 66 has the screenshot files located at:
http://grfcrawler.tt-forums.net/grf/66/75ttd.png
http://grfcrawler.tt-forums.net/grf/66/300ttd.png

The RSS feed contains the set's ID in its link, and it shouldn't be too hard to generalize from the above specific case to determine the URL for the images for an arbitrary set.

eis_os · Post by **eis_os** » 21 Jan 2007 11:14

It's hard, because in the ID dir you can have any filename you like...

If there is a valid way via rss xml to provide a image, let me know. I guess via an xml namespace it should be quite doable...

DaleStan · Post by **DaleStan** » 21 Jan 2007 11:17

Ah. So it is. I assumed that such non-obvious filenames meant that they were computer generated.
It is, however, quite possible to get a listing of all files in the ID dir.

eis_os · Post by **eis_os** » 21 Jan 2007 11:18

Yes, that may change however in the future. The prefect way is too put the link in the feed somehow... (as a dir can hold outdated files aswell)

-edit-
Where do you get the image from? Actually the rss you showed didn't has a proper namespace for that and it will be broken as soon as the feed description changes...

-edit2-

I added the namespace: http://search.yahoo.com/mrss/ now, and the description == content:encoded. I hope all links are ok that gets created, if something is wrong Andrex, please don't try to fix it yourself, but tell me.

I will add my own namespace for grfids soon aswell...

-edit3-
My current draft:
http://grfcrawler.tt-forums.net/rss/grf/

Maybe someday grfcrawler can collect and search feeds with this extension aswell...

Andrex · Post by **Andrex** » 21 Jan 2007 16:45

eis_os wrote: Where do you get the image from? Actually the rss you showed didn't has a proper namespace for that and it will be broken as soon as the feed description changes...

I get the image from the <content:encoded> section. That section is an HTML code which includes an image, and I simply parse it to retrieve the img source url and display it. The code might change, but as long as the image is there, the current PHP code I have will retrieve it.

Now, instead of dealing with TTDXC's MSXML parser, it would be much easier and secure to do this: create the GRFs list with PHP on the server, and TTDXC would simply show that page.

If you can install the Magpie RSS parser on your server (or any other server with PHP), that would be great, and I could make the script for the RSS list in minutes.

eis_os · Post by **eis_os** » 21 Jan 2007 17:25

Well using php to convert back a rss feed created by php and smarty is silly...

What page do you want or like? It should be no problem to list the last x changes in some kind of a layout...

The software GRF Crawler you see is fully written by me, so say me what you really need is better then patching x or y feature in the software in long run

The feed anyway already produces links to the pictures as well...

So, do you want a certain kind of webpage? A special layout?

-edit-
You don't meant something like this or?
http://grfcrawler.tt-forums.net/index.p ... t&limit=10

Andrex · Post by **Andrex** » 21 Jan 2007 19:52

Yes, something like that. I made a sample page with 3 items, the images link to the screenshot and the title to the proper page.

I included a javascript file used to create a small popup window for screenshots, the same could be used to show the GRF's page in a window no larger than 800x600 or 640x480.

If you want to save bandwidth, you can put only 5 or 6 items instead of 10. That also dependes on how often a new graphic appears there.

eis_os · Post by **eis_os** » 22 Jan 2007 20:00

Is the base target needed?
As example, I could without to much changes add a different style of output, however I am not that sure about the base target

Andrex · Post by **Andrex** » 22 Jan 2007 21:00

The javascript to open popups for screenshots must use a target of "_self", otherwise it will open a standard browser window. So you can change whatever you want as long as the script keeps working propery.

eis_os · Post by **eis_os** » 23 Jan 2007 08:38

As soon as I have time inbetween I will try to add it...

Andrex · Post by **Andrex** » 24 Jan 2007 01:56

OK, just let me know when you have the URL with the page. I hope it's sooner than later.

Transport Tycoon Forums

Request feature for the GRF Crawler site

Request feature for the GRF Crawler site

Who is online