SyntaxHighlightTools

Reading and writing XML is ugly. JSON is nicer for humans, but still it has much noise, which blurs meaning.

So there are SyntaxHighlightTools, which provide a simple language for defining syntax rules, and a simple tool for defining color themes.

tmLang

Language for defining syntax highlight rules.

Creating a new tmLang file

Create empty tmLang file

Run Command Palette and select "SyntaxHighlightTools: New tmLang File..."

Create tmLang file from tmLanguage file

1. Open tmLanguage file
2. Run Command Palette and select "SyntaxHighlightTools: New tmLang File..."

tmLang Language

Best is you take a look at :file:`tmLang.tmLang` for an example. Grammar you can obtain from :file:`tmLangTool.py`.

Here a quick overview on tmLang Language:

tmLang file has a header section, where you can define general settings of the language, and tokens, which you can use later in your regexes:

name      : Name of language
scopeName : source.my-lang
uuid      : <uuid generated from :code:`import uuid ; uuid.uuid4()`>
fileTypes : <first-suffix suffix2 suffix3>

Note

"<" and ">" in filetypes are part of syntax, syntax is leaned to word-list producer of perl6 syntax.

Continuing example with some tokens:

foo = some regex
bar = another regex using {foo}

As you can see here, you can directly use a token, after defining it.

Finally start your rule-repository, with the MAIN section (which is major patterns-list):

MAIN
    constant.numeric.integer.my-lang
        \d+

    #
    # includes are done with #name, where name can be
    #   1) a name of a collection in your rule repository
    #   2) a scope name like #source.python
    #   3) #$self for self reference
    #
    example.for.pattern.with.begin.and.end.pattern
        \{
        \}

        #included

    # if you list your patterns between your begin and end pattern
    # applyPatternLast is implied
    example.for.applyPatternLast
        \[

        #source.regex

        another.pattern.spec.here
            (fo(o))bar

            2: foo.bar
            # names the captured "o"

        \]

    example.for.nesting

        (?x) \(

        \)

        #$self

    # use "[]" for specifying an empty pattern list
    example.for.applyPatternLast.with.empty.patterns
        <
        []
        >


    example.for.long.regex
        (?x) this
            regex is very long

included

    ...

On Sublime Text Unofficial Documentation - Syntax Definitions you can find an example using JSON translation of PList Format. Here is the tmLang version of it:

name : Sublime Snippet (Raw)
scopeName : source.ssraw
fileTypes : <ssraw>
uuid      : ca03e751-04ef-4330-9a6b-9b99aae1c418

MAIN
    keyword.ssraw
        # Tab stops like $1, $2...
        \$(\d+)
        1: constant.numeric.ssraw

    keyword.ssraw
        # Variables like $PARAM1, $TM_SELECTION
        \$([A-Za-z][A-Za-z0-9_]+)
        1: constant.numeric.ssraw

    variable.complex.ssraw
        (\$)(\{)([0-9]+):

        1: keyword.ssraw
        3: constant.numeric.ssraw

        \}

        #$self

        string.ssraw
            .

    constant.character.escape.ssraw
        \\(\$|\>|\<)

    invalid.ssraw
        (\$|\>|\<)

If you want to add special behaviour to your syntax, you have to add (*.tmPreferences) and <language name>.sublime-settings files.

Syntax Specific Sublime settings

You can add any settings by simply:

Settings::
    translate_tabs_to_spaces: true
    draw_white_space: all

Please note, that it is tried to interprete value as json text. If not possible the stripped string value is taken. So the upper def results in:

{
    "translate_tabs_to_spaces": true,
    "draw_white_space": "all"
}

tmPreferences

Caution!

Using the following will override any already existing tmPreferneces file, with same name and also <languagefile>.sublime-settings, if you use the settings definition.

Note

There is no removal of any existing files, which are not named in the tmlang file.

Each tmPreferences definitions must contains scope and uuid.

The following table is taken from values from tmPreferences. It is unclear to me for some preferences, if they are evaluated by sublime text.

name	type
decreaseIndentPattern	regex
increaseIndentPattern	regex
disableIndentNextLinePattern	regex
unIndentedLinePattern	regex
bracketIndentNextLinePattern	regex
cancelCompletion	regex
symbolTransformation	subst program
indentSquareBrackets	regex
indentParens	bool
preserveIndent	bool
showInSymbolList	integer
disableDefaultCompletion	integer
TM_*	string
completions	list of strings
highlightPairs	list of strings
smartTypingPairs	list or strings

You find some information about these values in following sources:

• http://manual.macromates.com/en/appendix.html
• http://manual.macromates.com/en/preferences_items.html

And of course by analysing *.tmPreferences files.

Now here follow some examples. They are taken from real tmPreferences files. Starting with tmLang.tmlang file, there are already some proved examples. First define a comment character, e.g. used by ctrl+/:

Comment::
    uuid  : e07f5996-cef4-4a7c-9d2c-64e85be714ef
    scope : source.tmlang
    TM_COMMENT_START: #

For navigation using symbol list (ctrl+r), there are selected two scopes, with some little transformations:

Navigation - Pattern Sets::
    uuid: 62dfa0a4-51a9-4bc2-8cfc-adfea85d74e4
    scope: source.tmlang storage.type.class
    showInSymbolList: 1
    symbolTransformation:
        s/^/Pattern Set: /

Navigation - Scopes::
    uuid: 192ba968-fc92-433c-9a6a-7c4d521e486d
    scope: source.tmlang entity.name.function
    showInSymbolList: 1
    symbolTransformation: s/^/Scope: /

Three examples above create files Comment.tmPreferences, Navigation - PatternSets.tmPreferences and Navigation - Scopes.tmPreferences.

Maybe one last (not yet proven) example. It is taken from XML/Miscelleneous.tmPreferences:

Miscelleneous::
    uuid: 95788610-7E2E-45CE-9CCE-708FE0C90BF7
    scope: text.xml
    comment:
       /* Don't indentation
        *  <?, </, <!
        * <whatever></whatever>
        * <whatever />
        * <% %>
        * <!-- -->
        * <%-- --%>
        *
        * Do indent:
        *  <whatever>
        *  <%
        *  <!--
        *  <%--
        *
        * Decrease indent for:
        *  </whatever>
        *  -->
        *  --%>
        */
    decreaseIndentPattern: ^\s*(</[^>]+>|-->|--%>)
    highlightPairs:
        ( ) [ ] { } "\"" "\"" < >
    increaseIndentPattern:
        ^\s*<(([^!/?]|%)(?!.+?([/%]>|</.+?>))|[%!]--\s*$)

Note the highlightPairs. It is a list of strings, which can be either unquoted or quoted with double quotes. This is why the doublequote has to be quoted and escaped. Both highlightPairs and smartTypingPairs have to be lists with an equal number of elements, because they are automatically paired.

Not everything is yet tested properly in creation of tmPreferences files so I am pleased for feedback.

Extended Variables

tmTh

If you try to create a new color scheme, you quickly have the problem, that you do not know which scope-names to theme so you have to look around into other .thTheme files, which is very time-consuming.

Another thing is that there are no real norms about scope-names except in http://manual.macromates.com/en/language_grammars.html.

Creating a new tmTh file

1. Create a new empty window
2. Open tmLanguage files, you want to create color schemes for (or none for covering all language files)
3. Open tmTheme files, you want to use for your color defaults. Order is here important: the next overwrites the previous, if they have a definition for same scope name.
4. select "generate_tmTh" from Build Systems, go to one of the tmLanguage or tmTheme files and press F7 for running the build system.
5. A new view emerged containing the .tmth file. Save this file with your desired Color Scheme name.
6. Change back Build System to "Automatic".

Press F7 for generating your tmTheme file from tmth file.

Pitfalls

• Make sure you use ONE indentation style. There is made no difference between spaces and tabs, which may lead to strange errors.

Changes

2014-09-13

• you now can specify name and contentName for a rule like:

  meta.restructuredtext.link-explicit <string.other.ref.restructuredtext>
      # here come the rule
  

2014-01-31

• improved display of errors in tmLang2tmLanguage
• added file_regex in build system, such that you can jump to position of error

2014-01-16

• added tmPreferences definitions
• improved symbol list for navigation
• added completions, made them more context sensitive, such that completions list does not get overloaded
• added uuid completion, which creates always a new uuid.
• set whitespace to spaces per default

Author

Kay-Uwe (Kiwi) Lorenz <kiwi@franka.dyndns.org> (http://quelltexter.org)

Support my work on Sublime Text Plugins: Donate via Paypal