Apex Test Coverage Does Not Match?
November 22nd, 2011
Writing unit test in Salesforce is, in my opinion, a “black art”. There are so many workarounds and “techniques” that it makes the process extremely frustrating. The issue below makes me enjoy the process even less based upon the amount of time/money I spent wasting on it.
BTW, go vote on my idea to make Apex testing easier. You’ll thank me later.
I’ve seen a number of people on twitter recently comment that unit test coverage display different percentages when tests run in the browser vs. in Eclipse with “Run All Tests”. I just ran across this same issue and finally found the way to “work around” it.
I had an existing test class that I was updating and when I “Ran All Tests” from Eclipse I received a different result than when I “Ran All Tests” in my browser.
For some reason it looks like salesforce.com is caching the class and not re-running the actual test cases. When I edited the class, saved it and “Ran All Tests” again in the browser it finally displayed the same-ish results.
Categories: Salesforce
Apart from the difference in Test coverages shown in Browser and Eclipse, there’s one more between the coverage shown on the Class’ detail page & when you run its Test class individually.. Would be grateful if this is addressed too in the next release..
Amazing work with the blog by the way…
Thanks @Avinash!
Hi jeff,
Try this before you go and run all test cases
go to Develop –> Apex test Execution –> View History –> Clear Test Result
After this step all the catche history would be delete. Code coverage for all classes and triggers would be reset to 0%.
Now when you run all test after this you should get consistent results everytime.
I’ve been working pretty hard to make testing a better experience over the last several releases. Recent additions are things like:
1. Removing the restriction that test classes must be private, so you can put common test initialization code in non-test methods in test classes (and not require test coverage for your initialization code). It’s a poor man’s setUp().
2. The Apex Test Execution console page, which allows you to run individual test classes asynchronously.
3. Storing test results and code coverage in the DB, so you can view code coverage from the last time tests were run, without rerunning all the tests. The UI for browsing code coverage is faster, too. This is probably the feature that’s causing the issue you’re seeing, as you’re picking up data from previous test runs. Without knowing your specific setup, I can’t be sure.
4. Speaking of the code coverage UI, you can now view the code coverage from particular test methods. This lets you see which tests are covering which lines.
5. Made the code coverage calculate based on lines instead of statements (making it easier to figure out how we come up with the numbers we do). This brings us in line with other popular code coverage tools like Clover.
6. Exposing ApexTestQueueItem and ApexTestResult in the API and apex, allowing you to programatically run tests. You can set up nightly test runs this way with scheduled apex.
7. Coming in the next release (forward looking statements, yada yada): Data Silo. This allows you to write tests that only “see” data that was created as part of the test (for non-setup data types). This means that, if you’re creating packages, your tests are far more likely to succeed no matter where they’re installed. It also means that you can’t accidentally lock rows in your tests that might be needed by a production request.
Work in this area is ongoing. In particular, I want to make it easier to test web service integrations.
Sorry you’re having such frustrations. I’m the only developer working on this at the moment (and not full-time, at that), and I’d like to think that developing and running tests has gotten easier over the last year or so.
Great minds thinks alike
http://th3silverlining.com/2011/11/30/salesforce-different-percentage-code-coverage-in-different-environments/