FindInProject

Text search plugin for Sublime Text 3 projects.

This is an alternative to the default “Find in files” command that comes with Sublime Text. It includes an interactive result view and a configurable search thread that runs in the background.

Results are ordered using basic implementations of the TF-IDF and Page-Rank algorithms.

Installation

The plugin is tested on Windows and Linux but should also work on macOS. To install it from https://packagecontrol.io/ do the following:

1. Open the command palette and find “Package Control: Install Package”
2. Search for FindInProject and install.

To install from GitHub do the following:

1. Locate Sublime Text packages folder by choosing the menu:

Preferences -> Browse Packages...

1. Clone or download git repository into a new folder named “FindInProject” under the packages folder

Configuration

All configuration is available through the menu:

Preferences -> Package Settings -> FindInProject

This includes

• Default settings which can be copied into the user settings and then changed
• Default keymap which can overridden in the user keymap
• Default color scheme which can be copied into the user color scheme and then changed

The settings include options for

• Encodings to try
• Maximum line length in result view
• Directories and file extensions to ignore
• File sizes to ignore
• Excessive hit count (to cancel large searches)
• Term separation regex pattern (defaults to non-word characters such as whitespace and punctuation)
• Page reference regex pattern (defaults to double square brackets around word characters, for example [[SamplePage]])
• and more (descriptive comments are included in the settings file)

Usage

In normal contexts (using the default keymap) the following shortcut is available.

When in a result view (using the default keymap) the following shortcuts are available.

For details see the keymap file available through the menu:

Preferences->Package Settings->FindInProject* menu.

Search and result ordering

Multiple search and ordering algorithms have been incorporated. Each may be configured with different weightings etc.

Terms are extracted from pages and the search query as lower case entities. This includes splitting up camel case words into separate terms.

Algorithms: * TF-IDF Search * Page Name Search * Page Rank Ordering * Page Age Ordering

TF-IDF Search

The term frequency (TF) inverse document frequency (IDF) algorithm scans all files during the document indexing process and extracts from each file a set of terms. These terms are added to both the overall dictionary and the current document's dictionary and count the number of times each term is used. These counts are normalised to the total number of terms in document and inserted into the IDF map.

The search queries terms are used to calculate a score per document using:

doc_score = SUM (search_term.normal + document_term.normal) / len(overall_terms)
                FOR search_term in search_terms
                IF search_term in document_terms
                WHERE document_term = document_terms[search_term]

Each document term's score is checked against the threshold constant to determine if this document should be included in the results. The score is merged with the other search scores using a weighted average.

Related settings find_in_project_tfidf: * enabled - Whether this algorithm is enabled * verbose - Whether this algorithm logs at the debug or info level * weight - The document terms score weighting used in calculating the weighted average * threshold - The search term score threshold for inclusion in the results

Page Name Search

Each document's page name is split into terms and included in the search.

Related settings find_in_project_pagename: * enabled - Whether this algorithm is enabled * verbose - Whether this algorithm logs at the debug or info level * weight - The score weighting used in calculating the weighted average * threshold - The score threshold for inclusion in the results

Page Rank Ordering

The page rank algorithm identifies page references within each file where each page name is derived from the filename by removing the path and extension. Page references are (by default) just a page name surrounded by double square brackets.

Each file is scanned as part of the document indexing process and is added to the graph of page nodes with all page reference being added as links in the graph to other page nodes.

Once all files have been scanned then the page rank algorithm is calculates the page rank scores as follows:

DEF calculate_ranks:
    FOR page IN graph
        ranks[page] = (1 - damping) / len(graph) +
                        damping *
                            SUM ranks[in_page] / in_page.out_count
                                FOR in_page IN page.in_links

This is repeated using an iterative approach, where each iteration calculates the next set of ranks until sum of the deltas between successive rank scores is less than some specified threshold (epsilon):

ranks = MAP page: 1 / len(graph) FOR page IN graph

WHILE delta > epsilon
    WHERE delta = SUM abs(ranks[page] - prev_ranks[page])
        FOR page IN graph

    prev_ranks = COPY(ranks)
    calculate_ranks()
}

Each page rank score is merged with the other search scores using a weighted average.

Related settings find_in_project_pagerank: * enabled - Whether this algorithm is enabled * verbose - Whether this algorithm logs at the debug or info level * weight - The score weighting used in calculating the weighted average * damping - The page rank damping used in calculating the page rank in each iteration * epsilon - The page rank epsilon used in determining when to stop iterating

Page Age Ordering

Each document's last modified date used to order the results search.

Related settings find_in_project_age: * enabled - Whether this algorithm is enabled * verbose - Whether this algorithm logs at the debug or info level * weight - The score weighting used in calculating the weighted average

Mouse events

The current implementation of sublime allows simple mouse event mappings, however these mappings do not support selector contexts. As such, any mapping may conflict with other mappings defined.

The default handling of the double-click mouse event is to select and find the word double clicked. The command find_in_project_mouse_event has been added to provide support for selector contexts. This allows a custom mapping for the double-click mouse event to support both actions depending on selector context, for example:

File ${USER}/AppData/Roaming/Sublime Text 3/Packages/User/Default.sublime-mousemap:

[{
        "button": "button1", "count": 2,
        "command": "find_in_project_mouse_event",
        "args": { "commands": [
            {
                "command": "find_in_project_open_result",
                "run_for_selector": "text.findinproject"
            },
            {
                "command": "find_under_expand",
            }
        ]}
    }]

Releases

• 0.1.0: Initial version
  • Searches all files in project and displays results in separate buffer.
• 0.2.0: Enhanced search
  • Basic implementations of the TF-IDF and page-rank algorithms.
• 0.2.1: Usablity tweaks
  • Display query in result buffer.
  • Double-click to open result.
• 0.3.0: Extend search algorithms
  • Refactor search mechanism to better support multiple algorithms
  • Resolve double click issue in standard find in files results panel
  • Resolve page rank bug
  • Resolve import issue with unit tests
  • Add page name search
  • Add file age ordering