How we speed up testing and building process of our Ember app at Brandnew? ~2.5x faster!

The scale

  • project started 2 years ago
  • we pushed about 3 000 commits
  • let’s say 1.5 front-end developer was involved at the same time ;)
  • ~39k lines of application code (~46k with comments and empty lines) across ~1.3k files
  • 1 364 test cases (with ESLint)
  • 270+ components
  • 94 dependencies (that’s a bit scary)

Shipping fast is important

  • Running Tests — 10:26
  • Making whole build — 12:04
  • With deployment — 18:20

So every minute we gain in building process can gain us about 2.5 minutes in the whole process.

Run tests in parallel

$ ember exam --split=NUMBER_OF_PARTITIONS --parallel
$ ember exam --split=3 --partition=1 --parallel
# circle.ymltest:
override:
- npm run-script test -- --split=$CIRCLE_NODE_TOTAL --partition=`expr $CIRCLE_NODE_INDEX + 1`:
parallel: true
  • Running Tests — 3:19 (~3.1x faster)
  • Making whole build — 5:30 (~2.2x faster)
  • With deployment — 11:10 (~1.6x faster)
Timings per container
  1. Make decision which test should be run next at runtime and put it into available container. As far as I understand, this is not possible with ember-exam .
  2. Keep historical execution times for all tests and try to decide how to split them to make partitions more balanced.

Store tests execution time

# circle.ymltest:
override:
- npm run-script test -- --split=$CIRCLE_NODE_TOTAL --partition=`expr $CIRCLE_NODE_INDEX + 1` --silent -r "xunit":
parallel: true
# testem.js/* eslint-env node */
let options = {
test_page: 'tests/index.html?hidepassed',
disable_watching: true,
launch_in_ci: [
'Chrome'
],
launch_in_dev: [
'Chrome',
'Firefox',
'Safari'
],
framework: 'qunit'
};

if (process.env.CIRCLE_TEST_REPORTS) {
options.report_file = process.env.CIRCLE_TEST_REPORTS + '/junit/test-results.xml'
}

module.exports = options;
Test Summary in CircleCI

Update your machine

  • Running Tests — 2:50 (~3.7x faster)
  • Making whole build — 5:00 (~2.4x faster)
  • With deployment — 10:40 (~1.7x faster)

Update browser used for running tests

# circle.ymldependencies:
cache_directories:
- '~/downloads'
pre:
# download the latest Google Chrome if enabled by environmental variable $USE_LATEST_CHROME
- if [[ $USE_LATEST_CHROME == true ]]; then if test -f "$HOME/downloads/use_chrome_stable_version.sh"; then sh $HOME/downloads/use_chrome_stable_version.sh; else curl -o $HOME/downloads/use_chrome_stable_version.sh --create-dirs https://raw.githubusercontent.com/azachar/circleci-google-chrome/master/use_chrome_stable_version.sh && bash $HOME/downloads/use_chrome_stable_version.sh; fi; fi;
  • Running Tests — 2:40 (~3.9x faster)
  • Making whole build — 5:00 (~2.4x faster) — the same
  • With deployment — 10:40 (~1.7x faster) — the same

Let’s stop for a second

What’s next?

  • Incremental builds — like the ones you use on your dev. machine (cached per branch) — could be beneficial for feature branches.
  • Yarn instead of Bower and NPM — all dependencies are cached so it takes only 40 seconds to set everything up. I do not expect much progress here if yarn is truly faster.
  • Reduce number of Acceptance Tests with faster Integration Tests if possible — this is kind of work in progress 🛠. Maybe separate article with comparison would be helpful?

--

--

--

I connect humans and machines. Usually write about interfaces, digital products, and UX on tomekdev.com. Founder of checknlearn.com. A bit nerdy 🤓

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

RStudio and Databricks Better Together

{UPDATE} Bride Pony wedding girl princess dress up makeover Hack Free Resources Generator

Create a spring-boot application with DynamoDB and deploy it to EC2 instance with docker

How to achieve local High availability using Azure Availability sets , Role of Fault Domain &…

ICON P-Rep Projects

ICON P-Rep projects

🔗 Link Post: Not important enough: 1Password abandons its native Mac app

Parking Management System

Tailwind to Vanilla #2 — Two Shining Examples

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Tomek Niezurawski

Tomek Niezurawski

I connect humans and machines. Usually write about interfaces, digital products, and UX on tomekdev.com. Founder of checknlearn.com. A bit nerdy 🤓

More from Medium

4 Things to Consider When Starting a Sports OTT Service

Why Software Testing Services Are Relevant For Business | Blog

Bug Management | QA Perspective

Key Considerations for Choosing eKYC Solution