tensorflow unusual CUDA related error - python-2.7

Question

I've been using tensorflow for nearly two years and have never seen this one. On a new Ubuntu box, I have a fresh install of tensorflow in a virtualenv. When I ran a sample code, i got a Invalid Device error. It occurred when tf.Session() is called.
WARNING:tensorflow:From full_code.py:27: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
2017-06-05 11:01:55.853842: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 11:01:55.853867: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 11:01:55.853876: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 11:01:55.853886: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 11:01:55.853893: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 11:01:55.937978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with properties:
name: GeForce GTX 660 Ti
major: 3 minor: 0 memoryClockRate (GHz) 1.0455
pciBusID 0000:04:00.0
Total memory: 2.95GiB
Free memory: 2.91GiB
2017-06-05 11:01:55.938063: W tensorflow/stream_executor/cuda/cuda_driver.cc:485] creating context when one is currently active; existing: 0x19e5370
2017-06-05 11:01:56.014220: E tensorflow/core/common_runtime/direct_session.cc:137] Internal: failed initializing StreamExecutor for CUDA device ordinal 1: Internal: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_INVALID_DEVICE
Here is the full spec.
Ubuntu 14.04
CUDA 8.0
GeForce GTX 660 Ti
python 3.4.3

Answer 1

Thanks to someone from google, i figured out what went wrong. In this Dell box, there are two Nvidia graphic cards. First one comes with the manufacturer and is a NVS 310 card. As far as I know, this one does not have any compute capability and I never intend to use much of it.
I then added a second card, a GTX 660 Ti and I intended to use this one for all computations.
When Tensorflow is invoked, it defaults to Device 0, which is the NVS 310. And of course it throws an invalid error.
When I do this,
CUDA_VISIBLE_DEVICES=1 python myscript.py
it works.