.tests field

[jonathanong]

since apps are already split up into components, why not tests? i'm not 100% sure how this would work yet, but i want each component to be able to have its own tests:

{
  "name": "router",
  "dependencies": {}
  "development": {
    "dependencies": {}
  },
  "tests": [
    "tests/*.js"
  ]
}

then a test runner could go through each component (in resolution order) and run each file after building the component in development mode. thus, no more test/ folder!

we could add more structure to this like requiring the tests to be of the form:

test.js
test/*.js
tests/*.js

so it could be run easier outside of component (i.e. mocha cli). we can have a shortcut "tests": true for this.

[dominicbarnes]

+1 if we can also give test scripts the same CommonJS benefits.

[johntron]

I've already been using .tests with my test suites, so a big +1 here!

[jonathanong]

i'm thinking that this would expose a require('test') or something that once required will execute the tests. so component test would just run something like:

require('mocha');
require('test');

and everything will be executed. the builder will have to bundle the tests and dev deps correctly.

one thing we're going to run into is not being able to do require('assert') from outside the built as there won't be any more aliases. this would solve this issue as tests are included within the build, so it'll be able to resolve all these dependencies.

[tj]

SGTM

[tj]

either that or maybe since we're doing the nested env thing with .development maybe just having them in .development.scripts would make sense?

[jonathanong]

i was thinking that it would basically be .development.scripts but with an additional entry point (instead of index.js). otherwise, it's not totally clear which .development.scripts are the tests to the test runner. not totally sure yet. we could also be more strict and require tests to be in a test/ folder or something - that would work in .development.scripts, though i'd rather be explicit with tests = [stuff]

[jonathanong]

i'm now thinking that component build tests will create a standalone build with relevant js, css, and files with a test/index.html entry point. then we can publish this entire folder to gh-pages/test with a --branch gh-pages option, then going to component.github.io/emitter/test will run all the tests in the browser.

this might be a good way to do cross-browser testing, especially on an arbitrary system and browser. "can you go to this address and tell me what happens?" less opinionated than integrated saucelabs support and such.

i'm also thinking of integrating mocha since it's what 99% of components use.

what do you guys think?

[jonathanong]

also, doing saucelabs or whatever testing would be easier if we build everything into a small package.

[ianstormtaylor]

i like the idea of wrapping it up into a standalone. i think it should be it's own command though instead of being attached to build. the command could run build --dev internally i think, but lots of the other stuff is kinda opinionated for core

gh-pages might be problematic because people might want to use that for other things, so i wudnt want component testing to require clobbering pages

[jonathanong]

yeah i dunno. it could build automatically with build --dev but have a require('tests') entry point or something. component build tests just sounds the most semantic to me. maybe component build --tests

i was thinking --branch gh-pages --out test, which creates a test/index.html file, so people can designate a specific page just for tests. i'm not sure where to put these files in the current workflow. shouldn't clobber anything since you'll have to specify everything manually.