From:  "'chandrakant dhutadmal' via qa@mozilla.com" <qa@mozilla.com>
Date:  08 Apr 2016 12:23:39 Hong Kong Time
Newsgroup:  news.mozilla.org/mozilla.dev.quality
Subject:  

Re: Test Case Management

NNTP-Posting-Host:  63.245.214.181

Hi,
Is there any way in which one can take a look at TestRail system?  I would appreciate if someone has the instance deployed and could add me to the system.
My Email id is - cpdhutadmal@yahoo.com
Regards,Chandrakant Dhutadmal. 

    On Friday, April 8, 2016 3:05 AM, Axel Hecht  wrote:
 

 If there's need/opportunity: the Gurock brothers are in Berlin, and we 
have folks in Berlin, so maybe we'll be able to get a in-person 
conversation.

Re Stuart's

> Yeah fair points. We've been assessing it so far mostly for our needs,
> which don't include a lot of l10n testing. But it looks to have the
> flexibility in the test case meta data to assign tests to locales,
> versions, releases, etc.
I'm concerned about that from an l10n POV. I'd like to see practical 
experience.

With moztrap, we were in a similar boat, and I think we paid quite some 
price. Features like testing speed projections etc really need to be 
designed to support testing products that are localized.

Side-stepping, on the l10n side, we're quite happy about categorizing 
test results, and they're in a hierarchy of results. In l10n, a plain 
"yes/no" isn't good enough. 
http://www.qt21.eu/mqm-definition/issues-list-2015-12-30.html#issues_list 
is looking promising, in categorizing test failures. Those issue types 
at a corse level allow to understand our quality, and at a deeper level 
instruct us on how to improve and coach.

I'd personally really like to hear some war stories from the Gurocks or 
their clients in the field of l10n.

Axel

On 06/04/16 17:54, Stuart Philp wrote:
> My team has a free open-source license from gurock for 50 seats which we've
> been using to evaluate testrail, it is running here:
> http://testrail.stage.mozaws.net/
>
> We have also decided to go forward with trying it more permanently as our
> early feedback was mostly positive. We do however have a few questions we
> want to answer, you can see our discussion/questions if you're curious here:
>
> https://docs.google.com/document/d/1WkZMjYxEyEWHqmjKpE2idOLUCmQG6Fa5E6TQQiAL-BU/edit
>
> Once we have answered those questions we will move forward with planning
> how the transition will happen. I would note we think that if we were to
> ask for a larger seat count (tbd what that number would be, 100-500 would
> be a rough range for all QA, dev, and community needs), we should also look
> at paying for the service as it would no longer be a small side project but
> rather a large, key piece of QA operations at Mozilla.
>
>
>
> On Wed, Apr 6, 2016 at 11:38 AM, Lawrence Mandel 
> wrote:
>
>> Hi all,
>>
>> I would like to proceed with TestRail (or another system but from what I
>> gather this one seems to meet our needs). Specifically, I do not think that
>> we should continue to develop our own test case management tool in house.
>> Has anyone spoken with TestRail about their open source license? If not, I
>> volunteer Stuart and myself to do so.
>>
>> Lawrence
>>
>> On Wed, Apr 6, 2016 at 11:31 AM, Florin Mezei <
>> florin.mezei@softvisioninc.eu> wrote:
>>
>>> Hi guys,
>>>
>>> I think it was John who mentioned that we may be able to get a free
>>> license
>>> for Test Rail (based on this:
>>> https://blog.gurock.com/free-testrail-licenses-for-open-source-projects/
>>> ).
>>> Was wondering if anyone has contacted them to ask about this.
>>>
>>> I'm also adding Lawrence to this thread, as we've been discussing this
>>> internally, and our QA teams are very interested in getting this to
>>> replace
>>> MozTrap, gdocs.
>>>
>>> Regards,
>>> Florin.
>>>
>>> -----Original Message-----
>>> From: dev-quality
>>> [mailto:dev-quality-bounces+florin.mezei=
>>> softvisioninc.eu@lists.mozilla.org]
>>> On Behalf Of Delphine Lebédel
>>> Sent: Saturday, April 02, 2016 3:50 AM
>>> To: Stuart Philp 
>>> Cc: John Dorlus ; Axel Hecht ;
>>> qa@mozilla.com; Naoki Hirata 
>>> Subject: Re: Test Case Management
>>>
>>> It's been more than a year that I myself haven't executed l10n testing on
>>> Moztrap, so it's become a bit vague in my head. I'm sure QAnalysts and
>>> Marigold would have had good feedback for this - but I guess we can't
>>> really
>>> ask them anymore.
>>>
>>> What I do remember that is really important for l10n has been highlighted
>>> by
>>> Naoki and Stuart: the need for screenshots (really necessary) and the
>>> cited
>>> meta data. Something that would have been helpful (but not only from l10n
>>> point of view) would have been the ability to file bugs directly within
>>> moztrap, and not have to go back and forth between moztrap and bugzilla.
>>>
>>> I think I'd need to dig into the tool again to see if I can come with
>>> something else useful. Thanks for getting the conversation started :)
>>>
>>> delphine
>>>
>>> On Fri, Apr 1, 2016 at 5:08 PM, Stuart Philp  wrote:
>>>
>>>> Yeah fair points. We've been assessing it so far mostly for our needs,
>>>> which don't include a lot of l10n testing. But it looks to have the
>>>> flexibility in the test case meta data to assign tests to locales,
>>>> versions, releases, etc.
>>>>
>>>>
>>>>
>>>> On Friday, 1 April 2016, Naoki Hirata  wrote:
>>>>
>>>>> +1
>>>>>
>>>>> I hope that it's asystem that supports showing screenshots and
>>>>> comparisons as well.  I find that it helps for l10n testing as well as
>>> other things.
>>>>> Regards,
>>>>> Naoki
>>>>>
>>>>> On Fri, Apr 1, 2016 at 3:52 PM, Axel Hecht  wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> sorry for chiming in late. It'd be great if our test case
>>>>>> management supported running tests on localized versions, and had
>>>>>> some reporting support for that.
>>>>>>
>>>>>> CCing Delphine for the insights we gathered from our FxOS testing
>>>>> programs.
>>>>>> Axel
>>>>>>
>>>>>> On 25/02/16 00:30, John Dorlus wrote:
>>>>>>
>>>>>>> Greetings Mozillians,
>>>>>>>
>>>>>>> The quest for a test case management system has been a topic for a
>>>>> while
>>>>>>> now amongst several teams. Moztrap is a great tool but it's
>>>>> maintenance is
>>>>>>> slow and we have a need now to leverage someone else's code so we
>>>>>>> can focus on the new and shiny. There are many options out there
>>>>>>> and we want to
>>>>> find
>>>>>>> one that will support our Open Sourceness. We found a tools seems
>>>>>>> interesting and want to get some opinions on this or other
>>>>> alternatives.
>>>>>>> TestRail - TestRail is one of the most widely used Test Case
>>>>>>> Management systems amongst top tier tech companies. Comes with all
>>>>>>> of the bells
>>>>> and
>>>>>>> whistles (including integration with Bugzilla and Github) but is
>>>>> normally
>>>>>>> super expensive. I found a page on the TestRail site saying they
>>>>>>> give
>>>>> free
>>>>>>> licenses to open source projects.1
>>>>>>>
>>>>>>> Have any of you heard of something better? Better support? Better
>>>>>>> integration? Is TestRail a good choice? Let us know your thoughts.
>>>>>>>
>>>>>>> 1 -
>>>>>>>
>>>>> https://blog.gurock.com/free-testrail-licenses-for-open-source-projec
>>>>> ts/
>>>>>>> [image: photo]
>>>>>>> *John Dorlus*
>>>>>>> Senior Software Engineer in Test, Mozilla Corporation
>>>>>>> m:407.780.8031 | e:jdorlus@mozilla.org | w:http://www.mozilla.org
>>>>>>> |
>>>>> a:331
>>>>>>> E. Evelyn Avenue Mountain View, CA 94041 USA
>>>>>>>   <
>>>>> http://www.linkedin.com/in/johndorlus
>>>>>>>   <
>>>>> http://www.github.com/silne30>
>>>>>>>  
>>>>>>> 
>>>>>>> Get a signature like this: Click here!
>>>>>>> <
>>>>>>>
>>>>> http://ws-promos.appspot.com/r?rdata=eyJydXJsIjogImh0dHA6Ly93d3cud2lz
>>>>> ZXN0YW1wLmNvbS9lbWFpbC1pbnN0YWxsP3dzX25jaWQ9NjcyMjk0MDA4JnV0bV9zb3VyY
>>>>> 2U9ZXh0ZW5zaW9uJnV0bV9tZWRpdW09ZW1haWwmdXRtX2NhbXBhaWduPXByb21vXzU3Mz
>>>>> I1Njg1NDg3Njk3OTIiLCAiZSI6ICI1NzMyNTY4NTQ4NzY5NzkyIn0=&u=129278079350
>>>>> 478
>>>>>>>
>>>>>> _______________________________________________
>>>>>> dev-quality mailing list
>>>>>> dev-quality@lists.mozilla.org
>>>>>> https://lists.mozilla.org/listinfo/dev-quality
>>>>>>
>>>>> _______________________________________________
>>>>> dev-quality mailing list
>>>>> dev-quality@lists.mozilla.org
>>>>> https://lists.mozilla.org/listinfo/dev-quality
>>>>>
>>> _______________________________________________
>>> dev-quality mailing list
>>> dev-quality@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-quality
>>>
>>>
>>>

_______________________________________________
dev-quality mailing list
dev-quality@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-quality