I have TeamCity up and running in a mac environment. A trigger is also setup to to run a rake task containing a number of test cases on every git commit. The rake command successfully pulls the code and runs the test cases. The passing test cases are shown correctly in the TeamCity interface, however, the failing test cases do not show up and the build is marked as successful. When I look at the error Build Log, failing tests are shown as errors. The question is why TeamCity is interpreting failing tests as errors and not as actual failing tests?
The assertions are as simple as the following (obvious failure):
test 'simple_test' do
a = 14
ssert_equal 341, a
ends
PS: the check box to fail the build upon at least one test fail is turned on in Failure Conditions(i.e. at least one test failed)
Ruby version = 2.2.2,
Rails version = 4.2.1,
TeamCity version = 9.1.1,
Testing framework = minitest,
Here is the build log output
Teamcity is not picking up failed tests because mini-test reporter is not up and running. To fix this, following gems must be present in the Gemfile:
gem "minitest", :group => :test
gem 'minitest-reporters', :group => :test
Also minitest reporter must be called. The best place to call it is test_helper.rb
require 'minitest/reporters'
MiniTest::Reporters.use!
Related
If I have a .simplecov file, then when I run RAILS_ENV=test rake db:schema:load, I get a coverage report.
If I move the code from .simplecov to my test_helper.rb file (I use MiniTest), this does not happen.
I expect that this should never happen - loading the schema in the test environment is not a coverage test.
The reason this is annoying is that it causes my schema to load on CircleCI, generates a coverage below my threshold, and then fails the build.
I probably posted this question before a recent change to the simplecov readme... the change I needed to avoid this problem was to update my Gemfile to turn off auto-requiring of the simplecov gem:
gem 'simplecov', require: false
I then explicitly added Simplecov to my test/test_helper.rb as recommended in the README and that fixed this problem.
I am trying to run my rspec tests from rubymine, but getting this error:
...[BUG] Segmentation fault at 0x00000000000438
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
-- Crash Report log information --------------------------------------------
See Crash Report log file under the one of following:
* ~/Library/Logs/CrashReporter
* /Library/Logs/CrashReporter
* ~/Library/Logs/DiagnosticReports
* /Library/Logs/DiagnosticReports
for more details.
-- Control frame information -----------------------------------------------
c:0051 p:-17566078017732 s:0152 e:000151 TOP [FINISH]
c:0050 p:---- s:0150 e:000149 CFUNC :require
c:0049 p:0022 s:0146 e:000145 BLOCK /Users/Sunday/workspace/resilience/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri.rb:29
c:0048 p:0149 s:0143 e:000142 TOP /Users/Sunday/workspace/resilience/vendor/bundle/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri.rb:25 [FINISH]
c:0047 p:---- s:0141 e:000140 CFUNC :require
c:0046 p:0064 s:0137 e:000136 TOP /Users/Sunday/workspace/resilience/vendor/bundle/ruby/2.1.0/gems/loofah-2.0.2/lib/loofah.rb:3 [FINISH]
c:0045 p:---- s:0135 e:000134 CFUNC :require...
I have checked on google and followed some stack-overflow answers on how to solve this, but non seems to be working for me.
I have also tried to upgrade the rubymine debugger with the following commands: gem install ruby-debug-base19x --pre and gem install ruby-debug-ide --pre.
the first threw an error:
ruby_debug.c:861:98: error: no member named 'bp' in 'struct rb_control_frame_struct'
if (debug_context->frames[debug_context->stack_size - 1].info.runtime.bp <= thread->cfp->bp),
but the second was successful. Even then I still get the initial error on rubymine when I try to run my tests.
BTW: the tests run well from the terminal, and tests from other projects run well on rubymine. Only this perticular project is not running.
what can I do to make my rubymine run these rspec test from my projects. Thanks for any help.
I finally got it right. I have to go into Rubymine's preferences => Languages and Frameworks => Ruby SDK and Gems to update the version of ruby for the project from 2.1.2 to what is specified in the .ruby-version file for the project, which is 2.1.6.
After doing this, everything worked fine as expected.
I have successfully converted two projects from grails 2.2.4 to 2.4.3. I can run the tests, and run-app both from the grails command line, and within ggts.
Then, I converted it to a maven project, by using create-pom to get an initial pom.xml file. After getting the dependencies right, I can again run the tests and the app from the command line and ggts. But I am getting errors on my unit tests (not spock tests) using 'mvn test'.
The errors look like this:
java.lang.NullPointerException
at grails.test.runtime.GrailsApplicationTestPlugin.createParentContext(GrailsApplicationTestPlugin.groovy:143)
at grails.test.runtime.GrailsApplicationTestPlugin.initGrailsApplication(GrailsApplicationTestPlugin.groovy:96)
at grails.test.runtime.GrailsApplicationTestPlugin.onTestEvent(GrailsApplicationTestPlugin.groovy:327)
at grails.test.runtime.TestRuntime.deliverEvent(TestRuntime.groovy:295)
It feels like some kind of setup , or dependency issue.
I'm working on writing simple unit tests for a Rails 3 project, but I'm unable to actually execute any tests.
Case in point, attempting to run the test auto-generated by Rails fails:
require 'test_helper'
class UserTest < ActiveSupport::TestCase
# Replace this with your real tests.
test "the truth" do
assert true
end
end
Results in the following error:
<internal:lib/rubygems/custom_require>:29:in `require': no such file to load --
test_helper (LoadError)
from <internal:lib/rubygems/custom_require>:29:in `require'
from user_test.rb:1:in `<main>'
Commenting out the require 'test_helper' line and attempting to run the test results in this error:
user_test.rb:3:in `<main>': uninitialized constant Object::ActiveSupport (NameError)
The action pack gems appear to be properly installed and up to date:
actionmailer (3.0.3, 2.3.5)
actionpack (3.0.3, 2.3.5)
activemodel (3.0.3)
activerecord (3.0.3, 2.3.5)
activeresource (3.0.3, 2.3.5)
activesupport (3.0.3, 2.3.5)
Ruby is at 1.9.2p0 and Rails is at 3.0.3.
The sample dump of my test directory is as follows:
/fixtures
/functional
/integration
/performance
/unit
-- /helpers
-- user_helper_test.rb
-- user_test.rb
test_helper.rb
I've never seen this problem before - I've run the typical rake tasks for preparing the test environment. I have nothing out of the ordinary in my application or environment configuration files, nor have I installed any unusual gems that would interfere with the test environment.
Edit March 9th
Xavier Holt's suggestion, explicitly specifying the path to the test_helper worked; however, this revealed an issue with ActiveSupport.
Now when I attempt to run the test, I receive the following error message (as also listed above):
user_test.rb:3:in `<main>': uninitialized constant Object::ActiveSupport (NameError)
But as you can see above, Action Pack is all installed and update to date.
Edit March 13th
When attempting to run tests using rake test:units the following stack trace is dumped to the console:
test/unit/bookmark_test.rb:3:in `<top (required)>': uninitialized constant Objec
t::ActiveSupport (NameError)
from C:/Ruby192/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader.rb:5:in `load'
from C:/Ruby192/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader.rb:5:in `block in <main>'
from C:/Ruby192/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader.rb:5:in `each'
from C:/Ruby192/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader.rb:5:in `<main>'
rake aborted!
So looking into the file listed above, I see the following:
#!/usr/bin/env ruby
# Load the test files from the command line.
ARGV.each { |f| load f unless f =~ /^-/ }
To my knowledge, everything looks as expected.
Your test/test_helper file should have been created when you generated the application. It contains this valuable content:
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
fixtures :all
# Add more helper methods to be used by all tests here...
end
The second line here is the most important: it requires the config/environment.rb file at the root of your application, which in turn requires a lot of other things, including the valuable (I like that word today, ok?) ActiveSupport constant.
When you generate a controller, model or scaffold it'll also generate tests for those. I just ran rails g scaffold ticket in my app and it generated test/unit/ticket_test.rb which contains this:
require 'test_helper'
class TicketTest < ActiveSupport::TestCase
# Replace this with your real tests.
test "the truth" do
assert true
end
end
The first line of this file will require the test/test_helper.rb file that we jut saw. This will load ActiveSupport and the TestCase class within it, thereby making this test feasible. Everything else just flows on from there.
With all that explanation out of the way (even though it's something that you already know), I'm placing a large wager on it's something that's masscaring your LOAD_PATH, causing the test directory to be removed from it.
What's really unusual is that when you do specify the full path to the test/test_helper.rb you're saying it loads it, but ActiveSupport is still undefined. Well, that should be loaded as-per the description above. Is it actually loading config/environment.rb? Can you put something such as:
puts "LOADING CONFIG/ENVIRONMENT.RB"
At the top of your config/environment.rb file and then run the tests again? It should be output. Very unusual.
Continuing on the thread about LOAD_PATH... Got a dirty little secret you're not telling us about?
Actually, Dan Cheail makes a good point. You could be running the tests using ruby test/unit/ticket_test.rb in which case test_helper wouldn't be available, but still that still doesn't explain why when you specify the full path you're still getting an undefined constant ActiveSupport.
If you want to run a single test you should be doing this:
ruby -Itest test/unit/ticket_test.rb
That -I option there adds the test directory to the load path, meaning the test_helper file will be available through a straight require 'test_helper'. If it still errors after this, I reckon your test/test_helper.rb is either empty or broken.
The problem you're having is the way you're executing tests. Simply calling ruby test/unit/user_test.rb doesn't set up the load path, which explains the problems you've been having.
rake test:units is what you want and should work straight away.
Sorry about the post here, but I am unable to comment on questions yet.
what environment are you running, Win (has an issue with a .gemspec file) Linux, Mac?
Are you using RVM?
Test-Unit is installed by default with Rails, if you installed the gem Test-Unit you will get a conflict between the 2. try uninstalling the gem and your tests should start working.
If running on windows I would gem uninstall "autotest", then navigate to the following dir
drive:\Ruby192\lib\ruby\gems\1.9.1\specifications
In here you will find .gemspec files. Ensure that you dont have 2 autotest.gemspec files or any, for that matter. if so remove(delete) them, then download and gem install the autotest gem again. Grab the latest by using version switch.
You should be able to run your autotests. I did come across this once before, so to fix I simply removed the view and helper test files and wrote everything in the standard test file. Other than that I know running on windows autotest had issues, because of the ruby installer and bundler not clearing out things correctly and forgetting files.
I will find the link for you, to better explain.
Had been looking at the jsunit and jcoverage demos here (click on coverage report link. Open this in a new tab).
I was wondering if any one had done anything similar with Jasmine and JSCoverage ? I'm a little unsure on how to proceed.
[EDIT]
I am wondering if there is something I can do with a jasmine reporter. My Jasmine "hello world" example makes reference to a TrivialReporter. Maybe this can be extended ??
[EDIT]
I've wired up js-test-runner with jasmine right now. Now If I could think of a way to get coverage ??
If you're working on a ruby project and using jasmine via jasmine-gem, I have a patch that adds jscoverage support[1].
If you're using bundler, you can use this version of jasmine with the following command in your Gemfile:
gem 'jasmine',
:git => 'git://github.com/hjdivad/jasmine-gem',
:submodules => true,
:branch => 'jscoverage'
Make sure you've downloaded jscoverage and it's in your $PATH.
You can then add the following to jasmine.yml
coverage:
enabled: true
encoding: utf-8
tmp_dir: tmp
report_dir: public/coverage
skip_paths:
- public/javascripts/vendor
If this works for you, you may want to speak up on the pull request[2] to get it, or some variation, into jasmine-gem proper.
[1] https://github.com/hjdivad/jasmine-gem/tree/jscoverage
[2] https://github.com/pivotal/jasmine-gem/pull/37
If you're not using the jasmine-gem, or don't want to have to run a server to check coverage, I've written a gem that pulls together jscoverage and jasmine. It can run as a rake task in your CI builds.
It can be found here: https://github.com/firstbanco/jasmine-coverage
Install it, then just run
bundle exec rake jasmine:coverage
You're done.
EDIT: As the author of jasmine-coverage, I feel duty bound to to tell you about a better alternative: teaspoon. It requires more setup, but also allows running in the browser so you can use the Chrome debugger.