From:  Shu-yu Guo <>
Date:  26 Jan 2017 02:34:36 Hong Kong Time

Re: RFC: test262 integration in jstests


Woohoo, thanks for the CI work!

Outside of CI, I am strongly in favor running test262 by
default. Concretely, I am in favor of:
 - Default set to be everything, including test262, and
 - Ability to exclude entire subdirectories

They supersede many of our tests and is the general source of compliance
truth for many corner cases. The increased running time is worth it for
correctness when implementing new features. JIT hackers can most likely get
by most of the time without running the full suite.

On Wed, Jan 25, 2017 at 9:55 AM, Steve Fink  wrote:

> anba has done some major work to get test262 runnable as a CI test, and
> I've been looking into creating taskcluster jobs for it. One thing that
> could use input from a wider audience is how we want it to work with
> running jstests manually. The tests replace the js/src/tests/test262
> directory, which means that if we do nothing, the runtime of on
> an opt build goes from about 50 seconds to about 260 seconds on my machine.
> I haven't tried a debug build.
> One option is to suck it up. If everyone (but me) tends to use jit-tests
> for their smoke tests already, then it doesn't matter. And you can always
> explicitly choose subdirectories to run.
> Another option, which I've implemented but wanted some feedback before
> landing, is to say that running with no arguments gives you "the
> default set", rather than its current "everything". And that default set
> would be "all but test262". If you want everything, you can say
>   ./ $JS *
> But in order to implement that, I had to implement directory exclusions.
> Which means it wouldn't be much extra effort to implement
> --exclude=test262, in which case the first option is a little more
> tolerable: |./ $JS| would give you everything, and if you didn't
> want the huge pile of test262 tests, you could do |./ $JS
> --exclude=test262|.
> A third possibility that comes to mind is to have presets. |./
> $JS --smoke| or something.
> Opinions?
> _______________________________________________
> dev-tech-js-engine-internals mailing list