Download My Past Test Experience

My Past Test Experience
By Qingsong Yao
•
•
•
•
•
•
•
About Our Team
Katmai NewDateTime
Sparse Column
Dev10 SQL Express Setup
SQL Server Apollo
Dac In-place Upgrade
Dac Logical Import/Export
Our Team’s Engineering Excellent
• Family is more important than work.
• Encourage People to do innovation, spend time
on thinking and reading.
• What we did
–
–
–
–
–
Team Study Group
Tracking Time Spend every day
Weekly Matrix Afternoon
Root Cause Analysis
Test Robustness
• Reduce 30,000 meta tests with 12 testshell tests
• Merge many test suites into single test suites
Katmai Date and Time
• How we make decision?
– Always think in term of customer’s interests.
– Technique difficulty is never an excuse.
• How we make customer happy?
– .Net and SQL Server Fully Aligned
– Add new Built-in Function IsoWeek.
– Rewrite and reviewed BOL
• How we build our feature successfully?
– Build a team, then we will build the feature.
Sparse Column
• How we build the right features?
– Closely work with customer (Bi-weekly meeting, provide
early build for testing).
• Can we rely on partner for testing?
– No, SharePoint use our feature three years later.
• How dev improve the quality?
– QO dev write adhoc test, and is our bug bash winner.
• What is the effective way of finding bugs?
– Stress testing, SEI PipeLine, and TestQP
• How can we do better?
– Provide better weapon (testing library).
Dev10 SQL Express Setup
• What is the goal?
– Make sure user have the best setup experiences
• How we make decision?
– We take risk to make user experience better.
• How you do testing?
– Scenario based testing plan.
– Manual test and sign-off by China Vendor Team.
• Why we don’t do test automation?
– Spend more time on testing
– Find UI bugs.
– Use the feature as real user.
SQL Apollo
• How you do functional tests?
– Use Declarative Testing Idea, Focus on what, not how.
• How you do scenario based testing?
– Collection all possible DW database queries and running on the
lab for every build with no human involvement.
• How dev write tests?
– Modify TestQP to cover all new QE code paths.
• How to reuse existing tests?
– Parse and stored all queries in existing tests into central
database.
– Use the same way as scenario testing.
• How you do primtive performance testing?
– Use Perf Team’s Perf Test Automation Framework
– Measure CPU Instructions instead of CPU/Run Time
– Run Perf tests on the lab every week.
DAC In-place Upgrade
• What’s your testing methodologies?
– Functional; Model based; User Scenario;
Exploratory testing; Risk based testing; Manual
• How we take customer into account?
– We find many internal customers, and work with
them together.
– We did not make everyone happy in one release.
However, we committed to continue improving
our libraries.
DAC Logical Import/Export
• What is your testing strategy?
– Writing ATDD tests and testing as real user:
• Testing data is real
• Testing target is real.
• Do you have some demo?
Why we need DogFood by oureselve?
• Feel the pain of customer.
• We learn experiences from the beginning.
• Understand what is real user scenario and real
requirement of our customer.
• We get tests for free.
• Our sales can use the experiences as case study
to sale our product .
• We get the benefit of cloud. No more
deployment day any more.
Thanks
• My managers Jinghao Liu and Rajendra Gokhale
• My Colleagues:
Xin Zhang, Xuerong Tang, June Lou, Charlie Carson,
Aditya Sarpotdar; Anna Wawrzyniak;Charlie
Carson;Jason Short; Preetha Subbarayalu; Shireesh
Thota ;Renato Ferreira ; Alex Zhang; Frode
Kristensen; Gjorgji Gjeorgjievski; Jesus DeLaTorre;
Yiwei Zhang; Antony Zhong; Hongyi Wang; Hugh
Qu; Min He; Peng Li;Srini Acharya; Peter Carlin; Jun
Fang;Mostafa Elhemali: Dave Poole, Qingqing
Zhong; Artem Oks;Jinwen Wang; Yuelong
Jiang;Rami Koudsieh;