Test Coverage Integration?

Please provide us with the following information:

OS? Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)

Mac OS X (El Capitan)

Versions. Please run ng --version. If there's nothing outputted, please
run in a Terminal:
node --version
And paste the result here.

angular-cli: 1.0.0-beta.5
node: 6.0.0
os: darwin x64

(node 6.0.0)

Repro steps. Was this an app that wasn't created using the CLI? What change did you
do on your code? etc.

None - feature request.

The log given by the failure. Normally this include a stack trace and some
more information.

None - feature request.

Mention any other details that might be useful.

Angular-cli sets up Karma/Jasmine and basic unit tests which is really nice; thanks for that. However I'm wondering if there are plans to integrate code coverage reporting and threshold checking into the ng test flow (e.g. with Istanbul/Isparta, or similar).

In my company's starter kits we have a shortlist of things we consider mandatory for a production toolchain; the ability to fail CI when unit test coverage drops below a certain threshold is one of them.

Is code coverage on your radar?

For fun, I decided to try and get this working in a test project. I was able to get things working with karma-coverage and remap-istanbul.

Caveats:

There is no folder structure. There appears to be an issue already reported here: Missing folder structure in html output SitePen/remap-istanbul#50
I was not able to find a way to get the remap to auto-run after a karma run. I tried karma-remap-istanbul, but had no luck. This is probably due to user error. Also, it has an outdated version of remap-istanbul (Bump remap-istanbul version marcules/karma-remap-istanbul#7). I had used the pull request project instead of the npm package.
Using remap-istanbul requires you to list specific source files, which if you add multiple browsers, will be multiple and will be fragile due to the specific browser needed. This will require a script to gather and remap each of the files to overcome the CLI commands I used.

My modified karma.config.js for reference (omitted parts that didn't change):

    plugins: [
      require('karma-coverage'),
      require('karma-remap-istanbul'),
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-safari-launcher')
    ],
    preprocessors: {
      'dist/!(vendor)/!(*spec).js': ['coverage']
    },
    reporters: ['progress', 'coverage', 'karma-remap-istanbul'],
    coverageReporter: {
      type : 'json'
    },
    browsers: ['Chrome', 'Safari']

    // This didn't work for me. Kept saying "Couldn't find any specified files, exiting without doing anything."
    remapIstanbulReporter: {
      src: [
        'coverage/Chrome 50.0.2661 (Mac OS X 10.11.4)/coverage-final.json',
        'coverage/Safari 9.1.0 (Mac OS X 10.11.4)/coverage-final.json'
      ],
      reports: {
        html: 'coverage/report'
      }
    }

To get the remap working, I used the following. You may need to change the numbers up depending on your browser.

$ ng test --watch=false
$ ./node_modules/.bin/remap-istanbul -i coverage/Chrome 50.0.2661 (Mac OS X 10.11.4)/coverage-final.json -o coverage/Chrome 50.0.2661 (Mac OS X 10.11.4) -t html
$ ./node_modules/.bin/remap-istanbul -i coverage/Safari 9.1.0 (Mac OS X 10.11.4)/coverage-final.json -o coverage/coverage/Safari 9.1.0 (Mac OS X 10.11.4) -t html

After playing with it a bit more, I got this working on ng test and karma-remap-istanbul (from the PR version still). You can see the PR here: #895

Caveats:

There is still no file structure (same reason as above)
I assumed only 1 browser output allowed. This will probably need to have work done to support reports for multiple. The output is now in coverage/report.

Where did we land with this? I see that the issue is closed but I can't really tell what the conclusion is.

+1

Code coverage was added via #1455.

How can I run the code coverage tool? When I run "ng help" I can't tell what command I need to run?

It should run when you run ng test.

Actually it was changed to only run when doing ng test --code-coverage, because of how slow having it on all the time was.

Do you know, how to enable this automatically? So defining this parameter somehow in angular-cli.json or something like that. And how to force WebStorm to create coverage when I am using Karma task with the root karma configuration?

Is there a way to make the tests fail if the coverage falls before a threshold?

@cexbrayat

Hi @cexbrayat, what I wanted say is that I didn't get the percentage information after to configure as you explained. I mean, I didn't get the below information.

Low Coverage: GLOBAL X% of X% lines
Low Coverage: GLOBAL X% of X% statements
Low Coverage: GLOBAL X% of X% functions
Low Coverage: GLOBAL X% of X% branches

Well, It works on several of our apps, and it seems like some people 👍 it so I guess it should work for you too.

Read carefully my previous comment, and make sure you did not missed any step. You can also read the docs of the plugin https://www.npmjs.com/package/karma-istanbul-threshold

@cexbrayat

Hi @cexbrayat, I really appreciate your help.

I did it again step by step and I got the correct information. I also set up 'json-summary' and it provided everything I needed.

Thank you very much.

With @angular/cli: 1.0.0, there is no need to install any additional threshold plugin like karma-istanbul-threshold. As per cli documentation, just add below line and your are all set with threshold as well.

coverageIstanbulReporter: {
  reports: [ 'html', 'lcovonly' ],
  fixWebpackSourcePaths: true,
  thresholds: {
    statements: 80,
    lines: 80,
    branches: 80,
    functions: 80
  }
}

The thresholds property will enforce a minimum of 80% code coverage when the unit tests are run in the project.

Here is my output of

ng test --cc

16 04 2017 15:19:35.385:ERROR [reporter.coverage-istanbul]: Coverage for statements (66.28%) does no
t meet global threshold (80%)
16 04 2017 15:19:35.385:ERROR [reporter.coverage-istanbul]: Coverage for lines (63.8%) does not meet
 global threshold (80%)
16 04 2017 15:19:35.385:ERROR [reporter.coverage-istanbul]: Coverage for branches (44.55%) does not
meet global threshold (80%)
16 04 2017 15:19:35.385:ERROR [reporter.coverage-istanbul]: Coverage for functions (37.22%) does not
 meet global threshold (80%)

@khichar-anil that doesn't work for me, but using karma-istanbul-threshold works. I have no idea why it didn't work, so I'm sticking with what works

I wrote a terse, but perhaps helpful, blog post about this: How to Add a Test Coverage Report to an Angular CLI Project

I tried using thresholds with Angular CLI 1.0.1, but my tests don't fail even if I specify high threshold numbers.

ng test --code-coverage created code coverage for me.

@yfain

I agree with @yfain. It is not going to fail with Angular CLI 1.0.1.
It keeps getting success even I have all threshold 100%.

After upgrading the karma-coverage-istanbul-reporter to 1.2.1 it works as expected.

@yfain

Thank you @yfain, it is working now...

I can confirm. I updated karma-coverage-istanbul-reporter to 1.2.1 and now I can use the threshold property in the coverageIstanbulReporter config like so:

        coverageIstanbulReporter: {
            reports: ['html', 'json'],
            fixWebpackSourcePaths: true,
            thresholds: config.angularCli.codeCoverage ? {
                each: { // thresholds per file
                    statements: 80,
                    lines: 80,
                    branches: 80,
                    functions: 80
                }
            } : {}
        },

This example I made only applies thresholds when code coverage is enabled in the build, else it will use out of date coverage reports.

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

Navigation Menu

Repository navigation

Test Coverage Integration? #883

Please provide us with the following information:

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Test Coverage Integration? #883

Description

Please provide us with the following information:

Activity

delasteve commented on May 22, 2016

delasteve commented on May 23, 2016

jasonswett commented on Jul 19, 2016

CristyTarantino commented on Jul 27, 2016

filipesilva commented on Jul 27, 2016

seescode commented on Nov 17, 2016

delasteve commented on Nov 17, 2016

filipesilva commented on Nov 22, 2016

humpedli commented on Feb 1, 2017

schechter commented on Mar 1, 2017

3 remaining items

marciomsm commented on Apr 6, 2017

cexbrayat commented on Apr 6, 2017

marciomsm commented on Apr 6, 2017

anilkhichar commented on Apr 17, 2017

ajdaniel commented on Apr 26, 2017

jasonswett commented on Apr 26, 2017

yfain commented on May 4, 2017

aniruddhadas9 commented on May 8, 2017

SURENDARREDDY commented on May 9, 2017

yfain commented on May 9, 2017

SURENDARREDDY commented on May 10, 2017

ajdaniel commented on May 11, 2017

angular-automatic-lock-bot commented on Sep 7, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions