ctrl+shift+p filters: :st2 :st3 :win :osx :linux
Browse

Unit​Testing

by SublimeText ST2/ST3

Testing Sublime Text Packages

Details

  • 3 1.4.0
    0.10.6
  • github.​com
  • github.​com
  • 9 months ago
  • 2 hours ago
  • 6 years ago

Installs

  • Total 2K
  • Win 753
  • OS X 547
  • Linux 420
Sep 22 Sep 21 Sep 20 Sep 19 Sep 18 Sep 17 Sep 16 Sep 15 Sep 14 Sep 13 Sep 12 Sep 11 Sep 10 Sep 9 Sep 8 Sep 7 Sep 6 Sep 5 Sep 4 Sep 3 Sep 2 Sep 1 Aug 31 Aug 30 Aug 29 Aug 28 Aug 27 Aug 26 Aug 25 Aug 24 Aug 23 Aug 22 Aug 21 Aug 20 Aug 19 Aug 18 Aug 17 Aug 16 Aug 15 Aug 14 Aug 13 Aug 12 Aug 11 Aug 10 Aug 9
Windows 0 0 0 1 0 2 0 0 0 0 0 0 0 0 0 1 2 0 0 0 1 2 0 1 1 1 0 1 2 0 1 0 0 0 0 0 0 0 0 1 0 0 2 1 1
OS X 0 0 0 1 0 1 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 0 0 0 2 0 0
Linux 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 1 0 0 0 1 1 1 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0

Readme

Source
raw.​githubusercontent.​com

UnitTesting

CircleCI Build Status Build status codecov

This is a unittest framework for Sublime Text 3. It runs unittest testcases on local machines and CI services such as Travis CI, Circle CI and AppVeyor. It also supports testing syntax_test files for the new sublime-syntax format.

Preparation

  1. Before testing anything, you have to install UnitTesting via Package Control.
  2. Your package!
  3. TestCases should be placed in test*.py under the directory tests (configurable, see below). The testcases are then loaded by TestLoader.discover.

Here are some small examples

Running Tests Locally

UnitTesting can be triggered via the command palette command UnitTesting. Enter the package name in the input panel and hit enter, a console should pop up and the tests should be running. To run only tests in particular files, enter <Package name>:<filename>. <filename> should be a unix shell wildcard to match the file names, <Package name>:test*.py is used in default.

You could run the command UnitTesting: Test Current Package to run the current package. The current package will be first reloaded by UnitTesting and then the tests will be executed.

It is also possible to generate test coverage report via coverage by using the command UnitTesting: Test Current Package with Coverage. The file .coveragerc is used to control the coverage configurations. If it is missing, UnitTesting will ignore the tests directory.

Continuous Integration

Circle CI

To enable Circle CI Linux and macOS builds, copy the file .circleci/config.yml to your repository and change the environmental variable PACKAGE to the name of your package. Log in to Circle CI and add a new project.

Circle CI doesn't offer free macOS plan by default, but you could contact them for access if your package is open sourced:

We also offer the Seed plan for macOS open-source projects. Contact us at billing@circleci.com for access. If you are building a bigger open-source project and need more resources, let us know how we can help you!

Travis CI

To enable Travis CI Linux and macOS builds, copy the file: .travis.yml (caution: with a beginning dot) to your repository and change the environment variable PACKAGE to the name of your package. Log in to Travis CI to enable CI for your package..

AppVeyor CI

To enable AppVeyor Windows builds, copy the file appveyor.yml to your repository, change the PACKAGE variable in appveyor.yml. Log in to AppVeyor and add your repository as a new project.

Coverage reports

We support Codecov, Coveralls and Codacy. Codacov is slightly more favorable as it supports merging reports from different CIs.

Codecov

To submit coverage report to codecov.io:

  1. install codecov
  2. run codecov after success

Coveralls

To submit coverage report to coveralls.io:

  1. install python-coveralls
  2. run coveralls after success

Codacy

To submit coverage report to codacy.com:

  1. install both coverage and codacy-coverage

    pip install coverage codacy-coverage
    
  2. generate the xml report: coverage xml -o coverage.xml

  3. run python-codacy-coverage

Installing Package Control and Dependencies

If your package uses Package Control dependencies, you may want to install Package Control by uncommenting the line of install_package_control in Travis CI and AppVeyor configuration files.

Testing syntax_test files

Check this for an example.

Options

Use a different test directory

The default test directory is “tests”. To change the test directory, add a file unittesting.json to your repo with the corresponding directory name, eg unittest:

"tests_dir" : "unittest"

Redirect test result to a file

The test result could be redirected to a file by specifying the output variable in unittesting.json.

"output" : "foo.txt"

Deferred testing

Tests can be written using the Deferrable testcase, such that you are able to run sublime commands from your test cases and yield control to sublime text runtime and continue the execution later. Would be useful to test asynchronous codes.

An example would be found in here.

To activate deferred testing on Travis CI and AppVeyor, put the following line in unittesting.json.

"deferred": true,

PS: this idea was inspired by Plugin UnitTest Harness.

DeferrableTestCase is used to write the test cases. They are executed by the DeferringTextTestRunner and the runner expects not only regular test functions, but also generators. If the test function is a generator, it does the following

  • if the yielded object is a callable, the runner will evaluate the callable and check its returned value. If the result is True, the runner continues the generator, if not, the runner will wait until the condition is met.

  • If the yielded object is an integer, say x, then it will continue the generator after x ms.

  • Otherwise, the yield statement would yeild to any queued jobs.

Async testing

By default, the tests are running in the main thread and can block the graphic inference. Asychronized testing could be used if you need the interface to respond.

Async tests are usually slower than the sync tests because the interface takes time to respond but it is useful when there are blocking codes in the tests. An example would be found in here.

However, it is known that async test does not work very well with coverage. In general, it is recommended to use deferred testing over async testing since there is no need to worry about race condition.

To activate async testing on Travis CI and AppVeyor, put the following line in unittesting.json.

"async": true,

Note: if async is true, deferred is forced to be false (relaxation of this is in progress)

Others

Add Test Current Package build

It is recommended to add the following in your .sublime-project file so that c+b would invoke the testing action.

"build_systems":
[
  {
    "name": "Test Current Package",
    "target": "unit_testing_current_package",
  }
]

Credits

Thanks guillermooo and philippotto for their efforts in AppVeyor and Travis CI macOS support.