From:  Andy Sterland <Andy.Sterland@microsoft.com>
Date:  10 Jun 2014 13:23:16 Hong Kong Time
Newsgroup:  news.mozilla.org/mozilla.dev.js-sourcemap
Subject:  

RE: Adding checksums to source maps

NNTP-Posting-Host:  63.245.216.66

(Not sure if convention on the group is for inline commenting or top posting...)

Re: Dealing with normalized line endings
I’m not sure what to do in this case. Naively I was imagining that the checksum would be calculated without any extra transformation on the input to take into account line ending normalization. The checksum would take the input as a JavaScript string. In general I think the source map would either have to be explicit that a transformation happened or, by default, assume that none happened.

Though ultimately a consumer of the checksum could run the checksum with both normalized and not-so-normalized endings and if either match run with it. Though it kind of defeats the purpose of specifying it but having more permissive clients would still get the same end result as the chance of hash collisions would be so low but perf would be a pain as the combination of all the ‘permissive’ states could quickly get out of hand ☺.  So probably not a great idea.

Re: Checksum in comment
I think that ignoring end comments is something that F12 would do simply to be more permissive rather than something that the spec would even need to cover.

Re: Checksum in file
Having the checksum in the URL would help validate that a source map is for that file but I think we’d still need something to verify that the sources are for the source map. Though for our scenarios I think we still need to have something that is encapsulated in the source map. A solution that requires the developer to ship the comment at the end of the file would lock out developers who need to debug in cases where they can’t have comments in the file (say on a production server with comments stripped).

Re: Unrelated files
By unrelated I meant that an sourcemap chosen by a developer from the debugger may not be for the generated file or the source file. In the case of the generated file an example of where it would be broken is if the developer picked the sourcemap for a different version of the generated file. In the case of source files the most common broken, unrelated, use case is also likely to be mismatched versions. For example if the developer updated the generated file on the server but not the source files the relative paths might all resolve and files would be fetched but the source files wouldn’t match the actual source files. In both cases if the files don’t match it can be a very subtle issue for a developer to spot thus the need to programmatically verify the symbols are for the right files.

From: Brian Slesinsky [mailto:skybrian@google.com]
Sent: Monday, June 9, 2014 8:44 PM
To: John Lenz
Cc: Andy Sterland; dev-js-sourcemap@lists.mozilla.org; Ron Buckton
Subject: Re: Adding checksums to source maps

I wonder if it would make as much sense for the comment at the end of the JavaScript file to have the checksum of its sourcemap? The debugger would then use this as a key to look up the proper sourcemap in a content-addressable database somehow.

This would get us away from using URL's which is convenient in some ways; the sourcemap and source files can be stored in a private or public repository and the debugger configured to look up sourcemaps there.

But I'm not sure which use cases you're thinking about. What does "unrelated" when you say "a source map unrelated to either the source or generated files?" Surely there's some relationship or it wouldn't work.

- Brian

On Mon, Jun 9, 2014 at 7:22 PM, John Lenz > wrote:
One concern I raised  when this was discussed previously was whether or not
to ignore or normalize line endings when calculating the hash.  Currently
source maps are line ending agnostic.   And it would be good if it worked
for inlined scripts.

Ignoring ending comments is an interesting complication. I'm not sure what
I think about it.
On Jun 9, 2014 3:44 PM, "Andy Sterland" > wrote:

> One of the features we're adding to the IE F12 developer tools is the
> ability for developers to load a source map for any file in the debugger
> without the need for a comment in the file or in a http header. This would
> enable scenarios where developers have shipped generated JavaScript files
> to a server and either strip out comments or simply don't want to publish
> the sourcemap file and/or sources. Basically a mechanism to load symbols :).
>
> This feature would decouple the link between the generated file and the
> source map which makes it easier to load a source map unrelated to either
> the source or generated files. In cases where the developer has loaded the
> wrong source map we'd like to warn them in the debugger that the sources
> don't match so they can load another source map or carry on regardless. To
> do this we need to do know that the sources don't all line up which can be
> worked out by embedding a checksum for both the generated and source files
> in the source map which the consumer (F12) can them compare. Of course it
> would be up to the consumer and producers to do what they felt best and
> these properties would be optional. Right now we're looking at adding them
> to the TypeScript compiler and F12.
>
> Basically the proposal would add two new properties that would contain the
> checksums. The proposal below has them as Murmur 2 hashes but this is a key
> point to get feedback on especially if it should be something like SHA1,
> SHA2, MD5, CRC32 etc. Overall we choose Murmur v2 because it's fast, widely
> available, and used in the TypeScript compiler already.
>
> x_ms_fileMurmur2Hash - This property of type number would be the Murmur2
> generated hash for the file. This hash should match the entire contents of
> the file as received by the debugger. Consumers may take liberties with
> this to account for other build steps for example the hash could be of the
> entire file excluding any comments at the end of the file*.
>
> x_ms_sourcesMurmur2Hash - This property is an array of numbers each of
> which is an Murmur2 generated hash for a source file correlated by their
> indices in sources.
>
> Example:
> {
>   "version": 3,
>   "file": "combined.js",
>   "sources": ["a.ts", "b.ts"],
>   "names": ["method"],
>   "mappings": "AAAA,...",
>   "x_ms_fileMurmur2Checksum": 398506827,
>   "x_ms_sourcesMurmur2Checksum": [714841382, 473447668 ]
> }
>
> Thoughts?
>
> -Andy
>
> * It's a bit of a complication but our implementation in F12/VS needs to
> take a checksum of the generated file that excludes trailing comments at
> the end of the file. We need to do this mainly because some build systems
> like to add a signature at the end of JavaScript files as part of a signing
> process that happens after compilation. Thus the debugger that consume a
> source map and uses this feature will need to be tolerant and compare a
> checksum that excluded any trailing comments in the file (inc. the
> sourceMappingURL comment).
>
>
> _______________________________________________
> dev-js-sourcemap mailing list
> dev-js-sourcemap@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-js-sourcemap
>
_______________________________________________
dev-js-sourcemap mailing list
dev-js-sourcemap@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-js-sourcemap