NML for Visual Studio (1.0.1)

[PNDA_]

NML for Visual Studio

A new Project that I have released today. Before there was only a Highlighter for Notepad++ and here im expanding it to Visual Studio. It is planned to also be funcitonal with Atom Editor and Sublime Text, also planned is a Linter, to see Errors inside a file while editing it.
The current Highlighting does not support Stations, Canals, Bridges, Towns, Houses and Airports.
Have fun coding

Sample Code
.pnml file:

.lng file:

More Images can be found here: https://imgur.com/a/g8ds2ZT
In the current version (v1.0.1), #include does not get highlighted. That will return in the next update

Features
- Snippets
-> Currently 3 Snippets available for these Blocks: GRF Block, Item Block and Switch Block
- Full Syntax Highlighting
-> This includes .nml, .pnml and .lng

Credits
Code: PNDA
Sample Code: PNDA

License
- The License is GPL v2

Downloads:
It can be obtained from the Visual Studio Marketplace
The Source Code can be obtained here
All Releases can be found here
Or from here: (.vsix is inside the .zip)

[planetmaker]

Wow, great

Did you by chance generate it by means some script from parsing the NML source? The syntax highlighter for both, notepad++ and kate are generated by one.
If so, it would be great, if you could open a pull-request against NML so that it can be updated with every new build made for NML: https://github.com/OpenTTD/nml

[PNDA_]

Did you by chance generate it by means some script from parsing the NML source?

First of all thanks.
And no, I did not use a Script to generate this. I used .json Files to assign a colour to a variable, though it is Possible to do it in a .tmLanguage File, which works similarly to XML, used in the Notepad++ Highlighter. So maybe the Notepad++ Highlighter Script can be changed to fit the requirements of this File Type. Where are the scripts for creating the Notepad++ Highlighters located for me to find to maybe edit?

Just as an Example. this code gives the given variables (grf, param, property, graphics ...) the Blue Colour:

Code: Select all

{
	"name": "keyword.other.blocks.nml",
	"match": "(?<![_$[:alnum:]])(?:(?<=\\.\\.\\.)|(?<!\\.))(grf|param|item|property|graphics|spriteset|spritegroup|template|railtypetable|cargotable|basecost|sort|switch|date|bitmask|string|getbits|sound|import_sound)(?![_$[:alnum:]])(?:(?=\\.\\.\\.)|(?!\\.))"
}

Also, any Idea when a new Update with new Features of NML is coming along?

[planetmaker]

The scripts for generating the syntax highlight files are in the NML repo:

https://github.com/OpenTTD/nml/blob/master/gen_editor is used to call the generation of each file - which in itself are small python scripts for each: https://github.com/OpenTTD/nml/tree/master/nml/editors

As to features to NML: we shall see. There's no fixed timeline for anything. Pull requests are welcome.

[PNDA_]

So, thanks for sending me the links for editing the files.
Ive cloned the Repo, and after I figure out how to compile or create the output file I will edit "notepadpp.py" to fit to the needings of VS Code.

Note: I am quite bad at Python, no doubts.
So how would I use Python to create this file?
When trying to use it with "python vscode.py" I get this:

Code: Select all

Traceback (most recent call last):
  File "vscode.py", line 17, in <module>
    from nml.editors import extract_tables
ModuleNotFoundError: No module named 'nml'

When trying to do it with "setup.py" in the main directory:

Code: Select all

<python-directory>\dist.py:406: UserWarning: The version specified 
('unknown') is an invalid version, this may not work as expected with 
newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
  "details." % self.metadata.version
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: no commands supplied

Do I have to do it with the Makefile?

[planetmaker]

So, thanks for sending me the links for editing the files.
Ive cloned the Repo, and after I figure out how to compile or create the output file I will edit "notepadpp.py" to fit to the needings of VS Code.

Note: I am quite bad at Python, no doubts.
So how would I use Python to create this file?
When trying to use it with "python vscode.py" I get this:

Code: Select all

Traceback (most recent call last):
  File "vscode.py", line 17, in <module>
    from nml.editors import extract_tables
ModuleNotFoundError: No module named 'nml'

The module nml is specified by this repo. It is available (without installing it) only in the root directory of the repository. Note also, that you need python3 for this, not python2 (in case it should matter: usually python still refers to python2. Check by calling python -V.

When trying to do it with "setup.py" in the main directory:

Code: Select all

<python-directory>\dist.py:406: UserWarning: The version specified 
('unknown') is an invalid version, this may not work as expected with 
newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
  "details." % self.metadata.version
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: no commands supplied

Do I have to do it with the Makefile?

Neither is necessary. Just call gen_editor with the appropriate argument (currently kate or notepadpp). I created a dummy which adds visualstudio to the valid arguments.
It just needs modification of nml/editors/visualstudio.py to create the appropriate output: https://github.com/planetmaker/nml/comm ... 05fb9ad275
You can use that as a start and just edit the contents of visualstudio.py in that commit (which is a 1:1 copy of the kate.py file). With this diff call ./gen_editor visualstudio to generate the file
As for kate and notepad++ you can and probably want to make use of the keyword extractor already present in nml/extract_tables.py which provides convenient lists of the important words, separated by their type; theat is done in the def_write function at the bottom of the file.

As a side note:
setup.py is for building and bundling the package and indeed makefile is the way to build it for local use and call regression tests.
The generation of editor files is done neither in the makefile nor in setup.py. Thus you actually hit a soft spot: building the extensions is non-obvious and hidden in the devzone's build scripts. It probably could be added to both, setup.py and makefile. setup.py requires an argument as about what kind of distribution you want to build. like bdist or sdist.

As for python's/setup.py's complaint about the PEP440-non-conforming version string: yes. But that's intended. We want to use a version string similar to how OpenTTD uses now.

[PNDA_]

Thanks for the Dummy File. Im currently working on creating it to be compatible for VS.

First of all, VS (Code) takes a .tmLanguage file, which consist of PLIST Code, similar to XML and this file only gets called by "package.json" in the main folder of the extension, so the created file cannot be used by itself.

And Im finished inserting the code into the .py file, though im facing some problems:

Code: Select all

Traceback (most recent call last):
  File "gen_editor", line 4, in <module>
    from nml.editors import kate, notepadpp, visualstudio
  File "D:\Projects\PY\NML\nml\editors\kate.py", line 16, in <module>
    from nml.editors import extract_tables
  File "D:\Projects\PY\NML\nml\editors\extract_tables.py", line 16, in <module>
    from nml import unit, tokens, global_constants
  File "D:\Projects\PY\NML\nml\global_constants.py", line 1039
    'PLAIN_PLATFORM':                                   0x00;
                              ^
SyntaxError: invalid syntax

And im also unsure if what I've coded will work, therefore will not upload it to GitHub, yet.

This is parts of it currently:

Code: Select all

feature_text = """\
                <dict>
                    <key>match</key>
                    <string name="features">
                    <key>name</key>
                    <string>support.class.error.nml</string>
                </dict>
"""

def write_file(fname):
    handle = open(fname, "w")

    handle.write(header_text)
    handle.write(feature_text)
    handle.write("|".join(extract_tables.feature_names_table))
    handle.write(variables_text)
    handle.write("|".join(extract_tables.variables_names_table))
    handle.write(block_text)
    handle.write("|".join(extract_tables.block_names_table))
    handle.write(callback_text)
    handle.write("|".join(extract_tables.callback_names_table))
    handle.write(end_text)
    handle.close()

Ive only included one text, to not make this Post to long, but im trying to put the words listed in extract_tables.py into a <string></string> enclosure, and I hope that that what Ive coded does that, if not I'm happy if you could point it out to me

[planetmaker]

I wonder about that error you get... How do you call gen_editor or your python file and what python version do you use?

As to the name and format of the resulting file: use whatever it needs. Those two editors just happen to need rather plain xml files. If you are saying that it needs more than that one generated file: we sure could also include more, output then possibly to a separate sub-folder on generation, if it is needed to ship an appropriate extension for visualstudio in order to show syntax highlighting.

[Alberth]

There is no line with PLAIN_PLATFORM in current nml, which version are you using?

[PNDA_]

I wonder about that error you get... How do you call gen_editor or your python file and what python version do you use?

Ive got Python 3.7, and I call gen_editor like this:

Code: Select all

python gen_editor visualstudio

As to the name and format of the resulting file: use whatever it needs. Those two editors just happen to need rather plain xml files. If you are saying that it needs more than that one generated file: we sure could also include more, output then possibly to a separate sub-folder on generation, if it is needed to ship an appropriate extension for visualstudio in order to show syntax highlighting.

All Other Files are always the same, just for the README and Changelog of course.

There is no line with PLAIN_PLATFORM in current nml, which version are you using?

I copied the GitHub Repository of NML a few days ago, I think. Though recloning the repo makes it work again.

So after installing ply and Pillow and downgrading to Python 3.4, it now gives out a file
Now of to Bugfixing

Also, in notepadpp.py the Inserts are called with <Keywords name="features">, it doesnt seem to work if I use <string name="features" instead. I also would have to have a "\b" at the beginning and end of the list of variables and have to be seperated with a "|". (Ive got them to have the | in between already) Any ideas?

[planetmaker]

I wonder about that error you get... How do you call gen_editor or your python file and what python version do you use?

Ive got Python 3.7, and I call gen_editor like this:

Code: Select all

python gen_editor visualstudio

As to the name and format of the resulting file: use whatever it needs. Those two editors just happen to need rather plain xml files. If you are saying that it needs more than that one generated file: we sure could also include more, output then possibly to a separate sub-folder on generation, if it is needed to ship an appropriate extension for visualstudio in order to show syntax highlighting.

All Other Files are always the same, just for the README and Changelog of course.

There is no line with PLAIN_PLATFORM in current nml, which version are you using?

I copied the GitHub Repository of NML a few days ago, I think. Though recloning the repo makes it work again.

So after installing ply and Pillow and downgrading to Python 3.4, it now gives out a file
Now of to Bugfixing

Also, in notepadpp.py the Inserts are called with <Keywords name="features">, it doesnt seem to work if I use <string name="features" instead. I also would have to have a "\b" at the beginning and end of the list of variables and have to be seperated with a "|". (Ive got them to have the | in between already) Any ideas?

You can simply concatenate strings by adding them:

Something along the lines of

Code: Select all

start = "<string name="features>"
liststarts = "\b"
strg = start + liststarts + "|".join(extract_tables.feature_names_table) + liststarts
handle.write(strg)

maybe? Of course you can give the summation straight in the final call to handle.write.

[Alberth]

Also, in notepadpp.py the Inserts are called with <Keywords name="features">, it doesnt seem to work if I use <string name="features" instead. I also would have to have a "\b" at the beginning and end of the list of variables and have to be seperated with a "|". (Ive got them to have the | in between already) Any ideas?

Reading a few tutorial pages about strings in Python is likely the fastest way to get forward. Since Python is a popular language, you'll find many such pages without any problem, the official one is https://docs.python.org/3/tutorial/intr ... ml#strings but it may be too dense for you (depending on how much programming experience you have).

As for the strings, a string literal starts and ends with a quote character (either a single quote at both ends, or a double quote at both ends). To have a quote in the actual text (like your "features" text), use the quotes that you don't have in the text, ie

Code: Select all

'<Keywords name="features">'

. An alternative is to escape the quotes (ie tell Python, it's not the end of the string), like

Code: Select all

"<Keywords name=\"features\">"

The backslash is the escape character which says to Python (for escaped quotes) 'the next character is a normal character rather than something special', so it continues reading more string characters rather than stopping at the first " in your original version. To get a \ character in the string, you have to escape the \, and you get "\\". You can use that for eg

Code: Select all

"\\b"

which means the text \b .

[PNDA_]

Thanks for your help planetmaker and Alberth. Very appreciated

I just did a Pull Request on GitHub to merge my Branch with the new visualstudio.py Script.
Also, i found some reference online to .tnml files, what are those?

NML for Visual Studio 1.0.1

- All Variables, Callbacks and Blocks are now featured and working with this Update in both .nml and .pnml files
- #include Highlighting is temporary removed
- LNG Files now have full Highlighting Support for ##grflangid, ##gender, ##map_gender and everything else.

Downloads
Download from the Visual Studio Marketplace
Or from here:

nml-language-1.0.1.zip

(12.72 KiB) Downloaded 349 times

(.vsix installer is in the .zip)

[zeentch]

I know this is an extremely noobish question but i've just started looking into NML coding and trying to learn it since i want to start making some new trains for OTTD and so highlighting i think would help a lot, since i'm using Sublime Text i was directed here and with the file downloaded, i have no clue how to make them work with sublime text, if i try to make a new syntax file for sublime text it's file ending is *.sublime-syntax and the zip doesn't contain this and i have no idea how to "convert" it to it or how to make it work with what is in the folder.

Could someone maybe explain how i can do it or prepare an .sublime-syntax file for nml, pnml & lng file endings?

Thanks a bunch for this and hope i'm not to troublesome.

PS. Learning NML on my own from know virtually no other programming languages is quite hard, been looking through tutorials for a week and still only know how to do like 4 lines so far xD haha DS.

[PNDA_]

I know this is an extremely noobish question but i've just started looking into NML coding and trying to learn it since i want to start making some new trains for OTTD and so highlighting i think would help a lot, since i'm using Sublime Text i was directed here and with the file downloaded, i have no clue how to make them work with sublime text, if i try to make a new syntax file for sublime text it's file ending is *.sublime-syntax and the zip doesn't contain this and i have no idea how to "convert" it to it or how to make it work with what is in the folder.

Could someone maybe explain how i can do it or prepare an .sublime-syntax file for nml, pnml & lng file endings?

Thanks a bunch for this and hope i'm not to troublesome.

PS. Learning NML on my own from know virtually no other programming languages is quite hard, been looking through tutorials for a week and still only know how to do like 4 lines so far xD haha DS.

Hey zeentch,
The .zip you download from here is not meant to be used for Sublime Text rather with Visual Studio or Visual Studio Code. I personally don't use Sublime nor do I know how to make a syntax for it, you would have to look up a tutorial for that or maybe another forum member can help you. There probably is no very easy way to just convert the syntax files.
Also if you want to learn NML here's a tip:
I learn languages really quick if I look at some code. There are a few github repositories with some code for trains/industries/roadvehicles. You can also look on openttdcoop.org for a few examples (https://dev.openttdcoop.org/projects).

[PNDA_]

Wow, it's been a long time.

But an important release just happened. To supplement the release of NML 0.6 I have released version 1.2.2.
The accumulated changelog of all recent versions include:

• Updated to NML 0.6
• All new syntax highlighting system (better compatibility with themes)
• Better support for numeric constants and strings
• In-editor NML documentation (Hover over keywords like 'switch' to see a short snippet from the NML wiki)
• Added comment support for LNG files
• Added color support for LNG files

You can get it on the marketplace or just by search NML/NewGRF in the VSCode extension search.
https://marketplace.visualstudio.com/it ... l-language

[Tomplus]

It already exists for Sublime syntax highlighting system, it is:
Neko NML Syntax, https://packagecontrol.io/packages/neko%20nml%20syntax
https://github.com/R32/sublime-nekoml

[qamil]

Hi,
just after installing it into VS Code I have noticed weird comments highlighting related to '#' sign. First of all, #define and #include looks like whole line after this is marked as comment.

includes.png (13.05 KiB) Viewed 7865 times

Second case is some weird combination of #define and passing arguments to this macro. Again, after noticing one # sign rest of line is marked as comment, even if it should be marco concatenation. And also, as a result, we are getting some incorrectly coloured braces

macro1.png (5.05 KiB) Viewed 7865 times

macro2.png (30.25 KiB) Viewed 7865 times

But, on the other hand, great extension!

Regards,
qamil