I have a windows application that does a bulk copy of data to an SQL table, using the bcp_xxx() functions and a standard "SQL Server" ODBC connector.
I want to move to supporting native SQL client connections
According to what I have seen, all I need to do is add
#define _SQLNCLI_ODBC_
#include <sqlncli.h>
to the top of the cpp file, and link against "sqlncli10.lib" rather than "odbcbcp.lib."
However, when I call bcp_bind(), I get "Access violation reading location 0xffffffffffffffff"
This happens with both "SQL server" and "SQL Native Client" ODBC connections.
The code being executed is:
SQLHANDLE hEnv;
SQLHANDLE hConnection;
LPCBYTE data = (LPCBYTE)malloc(100);
SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &hEnv);
SQLSetEnvAttr(hEnv, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, SQL_IS_INTEGER);
SQLAllocHandle(SQL_HANDLE_DBC, hEnv, &hConnection);
SQLSetConnectAttr(hConnection, SQL_COPT_SS_BCP, (void *)SQL_BCP_ON, SQL_IS_INTEGER) ;
SQLConnect(hConnection, "DSN", SQL_NTSL, NULL, 0, NULL, 0);
bcp_init(hConnection, "DB_TABLE", NULL, NULL, DB_IN);
bcp_bind(hConnection, data, 0, SQL_VARLEN_DATA, (LPCBYTE)"", 1, SQLCHARACTER, 1);
The problem appears to be due to using "SQL server" ODBC connections at the same time as the bulk copy routines are trying to use native connections.
If I change all other ODBC connections to be SQL Native, the bulk copy routines work correctly.
Related
I am trying to establish a connection using libssh(https://www.libssh.org/) to a machine with SUSE Enterprise operating system installed.
ssh_options_set(tempSshSession, SSH_OPTIONS_HOST, host2Connect);
ssh_options_set(tempSshSession, SSH_OPTIONS_LOG_VERBOSITY, &verbosity);
ssh_options_set(tempSshSession, SSH_OPTIONS_PORT, &port);
int rc = ssh_connect(tempSshSession);
if(rc != SSH_OK) {
printf("Error connecting to the localhost\n");
exit(-1);
}
// Authorized by the password.
rc = ssh_userauth_password(tempSshSession, username, password);
After ssh_userauth_password() call it returns the response : "SSH_AUTH_DENIED: Authentication failed: use another method"
It works fine for operating systems like: Ubuntu, Linux Generic and CentOS.
Thanks
Check /etc/ssh/sshd_config. You should find PermitRootLogin forced-command-only somewhere there. Set that to yes and your problem should be gone. Please keep in mind that there is a reason, why folks have decided to not allow per default root user/pass logins.
I have a C/C++ DLL that is connecting to SQL and issuing a large number of ODBC queries rapidly in a loop. The only thing is that it is turning out to be so much slower from the ODBC DLL than running the query from T-SQL in Management Studio. Many orders of magnitude slower.
At first I thought it might be the query itself, but then I stripped it down to a simple "select NULL" and still got the same results.
I was wondering if this is expected or whether there is some ODBC setting that I am missing or getting wrong?
First I connect like this (for brevity I have omitted all error checking, however, retcode is returning SQL_SUCCESS in all cases):
char *connString = "Driver={SQL Server};Server=.\\ENT2012;uid=myuser;pwd=mypwd";
...
retcode = SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &henv);
retcode = SQLSetEnvAttr(henv, SQL_ATTR_ODBC_VERSION, (void*)SQL_OV_ODBC3, 0);
retcode = SQLSetEnvAttr(henv, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, SQL_IS_UINTEGER);
retcode = SQLAllocHandle(SQL_HANDLE_DBC, henv, &hdbc);
SQLSetConnectAttr(hdbc, SQL_LOGIN_TIMEOUT, (void*)5, 0);
retcode = SQLDriverConnect(
hdbc,
0,
(SQLTCHAR*) connString,
SQL_NTS,
connStringOut,
MAX_PATH,
(SQLSMALLINT*)&connLen,
SQL_DRIVER_COMPLETE);
Then I prepare the statement, bind a parameter (unused in this example), and bind a column like this:
char queryString = "select NULL;";
SQLLEN g_int32 = 4;
SQLLEN bytesRead = 0;
...
retcode = SQLAllocHandle(SQL_HANDLE_STMT, hdbc, &hstmt)
retcode = SQLPrepare(hstmt, queryString, SQL_NTS);
retcode = SQLBindParameter(hstmt, 1, SQL_PARAM_INPUT,
SQL_C_LONG, SQL_INTEGER, sizeof(int), 0, spid, 0, (SQLLEN*)&g_int32))
retcode = SQLBindCol(hstmt, 1, SQL_C_CHAR, col_1, 32, &bytesRead);
Finally, I repeatedly call the query (e.g., 10000 times) in a loop like this:
retcode = SQLExecute(hstmt);
retcode = SQLFetch(hstmt);
SQLCloseCursor(hstmt);
This takes about 90 seconds to run 10000 times in the ODBC DLL. Testing on a 4 core Windows 2008 R2 Server running SQL 2012 x64.
On the other hand, if I run, what looks to me to be, an equivalent test in Management Studio, it takes less than a second:
declare #sql varchar(128), #repeat int;
set #repeat = 10000;
set #sql = 'select NULL;';
while #repeat > 0 begin
exec(#sql);
set #repeat = #repeat - 1;
end;
Can someone point out something that I am overlooking? Some flaw in my logic?
Thanks.
Neil Weicher
www.netlib.com
This is too long for a comment
declare #sql varchar(128), #repeat int;
set #repeat = 10000;
set #sql = 'select NULL;';
while #repeat > 0 begin
exec(#sql);
set #repeat = #repeat - 1;
end;
does not realistically simulate the same as 10000 remote calls. exec bypasses a lot of the internals of setting up a request. To simulate 10000 calls do this in SSMS:
select NULL;
go 10000
and measure. Output as text probably should be used to avoid timing around SSMS grid display.
I'm not all that familiar with the T-SQL stuff but here are a couple things to consider.
Your ODBC driver has to transfer data via your network and I suspect the T-SQL execution does not. Next to disk IO, transfering data on the network is one of the slowest things your ODBC driver will have to do. You may find that the driver is spending considerable time either waiting on the data to travel to clearing data off the wire.
Also, it's not clear to me that your T-SQL example actually moves and data but your ODBC example does when SQLFetch is called. The T-SQL may just be executing the query and never fetching any data. So, removing SQLFetch from the loop might be a more equal comparison.
To see if data transfer is your limiting factor estimate how much data will be included in all the records you fetch with ODBC and try to move that much data between the two machines with something like FTP. An ODBC driver will never be able to fetch data faster than a simple raw transfer of data. I see you are just fetching NULL so not much in your result set but the driver and database still transfer data between them to service this request. Could be several hundred bytes per execution\fetch.
I faced the same issue. I solved it by changing the "Cursor Default Mode" setting of the DSN for the ODBC driver (through the ODBC Administrator tool) from "READ_ONLY" to "READ_ONLY_STREAMING". This alone increased the speed of my application (query data and write them to file) from 260 seconds to 51 seconds using Java 32-bit and from 1234 seconds to 11 seconds using C++.
See this post: http://www.etl-tools.com/forum/visual-importer/1587-question-about-data-transformation-memory-usage?start=6
I have a DB2 database in my CentOS 6.5 64 bit machine and have installed ODBC drivers to access the database from my C++ application.
I am using SQL CLI APIs to fetch the data from the database.
If I fetch the data from my main() function of the C++ application, SQL CLI APIs work fine (e.g. SQLAllocHandle) and I am able to read/write data from the database.
If I try to do the same operation on my POSIX thread, SQL CLI APIs fail to initialize the handle and eventually read/write fails.
SQLRETURN sqlrc = SQL_SUCCESS;
SQLCHAR pszSqlState[100];
SQLINTEGER pfNativeError[100];
SQLCHAR pszErrorMsg[100];
SQLSMALLINT cbErrorMsgMax;
SQLSMALLINT pcbErrorMsg;
/* allocate an environment handle */
sqlrc = SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &m_hEnv);
if (sqlrc != SQL_SUCCESS)
{
return 1;
}
sqlrc = SQLGetDiagRec(SQL_HANDLE_ENV, m_hEnv, 1, pszSqlState, pfNativeError, pszErrorMsg, 100, &pcbErrorMsg);
/* set attribute to enable application to run as ODBC 3.0 application */
sqlrc = SQLSetEnvAttr(m_hEnv,
SQL_ATTR_ODBC_VERSION,
(void*)SQL_OV_ODBC3,
0);
/* allocate a database connection handle */
sqlrc = SQLAllocHandle(SQL_HANDLE_DBC, m_hEnv, &m_hDBconn);
sqlrc = SQLGetDiagRec(SQL_HANDLE_DBC, m_hEnv, 1, pszSqlState, pfNativeError, pszErrorMsg, 100, &pcbErrorMsg);
/* connect to the database */
sqlrc = SQLConnect(m_hDBconn,
(SQLCHAR *)db1Alias, SQL_NTS,
(SQLCHAR *)user, SQL_NTS,
(SQLCHAR *)pswd, SQL_NTS);
//sqlrc = SQLGetDiagRec(SQL_HANDLE_DBC, m_hDBconn, 1, pszSqlState, pfNativeError, pszErrorMsg, 100, &pcbErrorMsg);
return sqlrc;
I am using Eclipse / IBM Data Studio for development.
I googled for known issues without any luck.
How to access a DB2 database with ODBC from a separate thread?
Info :
The first APi
SQLAllocHandle( SQL_HANDLE_ENV, SQL_NULL_HANDLE, &m_hEnv )
returns SUCCESS but the value of m_hEnv is invalid (-ve value).
So the subsequent APIs return -2 which is SQL_INVALID_HANDLE .
Call to
SQLSetEnvAttr(m_hEnv,SQL_ATTR_ODBC_VERSION,(void *)SQL_OV_ODBC3, 0);
returns -2 SQL_INVALID_HANDLE .
The problem was, my application which uses SQL CLI code was not compiled using -D_REENTRANT flag.
Now I am able to use the SQL APIs inside my threads. Thanks all for your inputs.
The relevant Microsoft doc is:
Blocking Direct Write Operations to Volumes and Disks
CreateFile, remarks on Physical Disks and Volumes
The executable is written in C++ and it calls CreateFile() to open an SD card that has no filesystem. The CreateFile() and consecutive ReadFile() calls are successful for GENERIC_READ without Administrator privileges.
CreateFile fails for GENERIC_WRITE even with Administrator privileges. In the explorer, I set Run as Administrator under Properties > Compatibility > Privilege Level. I also tried to run the executable from an Administrator cmd (started with Ctrl+Shift+Enter, "Administrator:" is in the window title, properly elevated). Still, I get ERROR_ACCESS_DENIED (0x5).
Do I have to pass something else to CreateFile? I have no idea what security attributes are, I just pass NULL, relevant code is here at line 92, and here at line 48.
Or is there anything else that should be set to run the process with Administrator privileges?
A related questions:
Can I get write access to raw disk sectors under Vista and Windows 7 in user mode?
Raw partition access in Windows Vista
How to obtain direct access to raw HD data in C?
Is there a clean way to obtain exclusive access to a physical partition under Windows?
While the answer of #MSalters makes sense, it is not how my code works. In fact it is so counter-intuitive, I spent several days making sure the code does in fact work.
These code snippets are in a proven, mass consumer market software product. When it needs to modify an on-disk structure, it dismounts the win32 volume so it can modify NTFS or FAT filesystem structures. Interestingly, the volume access handle is read-only:
char fn [30];
snprintf (fn, sizeof fn, "\\\\.\\%s:", vol -> GetVolName ());
vol_handle = CreateFile (fn, GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE, NULL,
OPEN_EXISTING,
FILE_FLAG_NO_BUFFERING | FILE_FLAG_RANDOM_ACCESS,
NULL);
if (vol_handle == INVALID_HANDLE_VALUE)
{
// show error message and exit
}
If unable to get write access to a volume or partition, this code forces a dismount if the user authorizes such after a stern warning:
if (!DeviceIoControl (vol_handle, FSCTL_DISMOUNT_VOLUME,
NULL, 0, NULL, 0, &status, NULL))
{
DWORD err = GetLastError ();
errormsg ("Error %d attempting to dismount volume: %s",
err, w32errtxt (err));
}
// lock volume
if (!DeviceIoControl (vol_handle, FSCTL_LOCK_VOLUME,
NULL, 0, NULL, 0, &status, NULL))
{
// error handling; not sure if retrying is useful
}
Writing is then fairly straightforward, except for positioning the file pointer by 512-byte sector:
long hipart = sect >> (32-9);
long lopart = sect << 9;
long err;
SetLastError (0); // needed before SetFilePointer post err detection
lopart = SetFilePointer (vol_handle, lopart, &hipart, FILE_BEGIN);
if (lopart == -1 && NO_ERROR != (err = GetLastError ()))
{
errormsg ("HWWrite: error %d seeking drive %x sector %ld: %s",
err, drive, sect, w32errtxt (err));
return false;
}
DWORD n;
if (!WriteFile (vol_handle, buf, num_sects*512, &n, NULL))
{
err = GetLastError ();
errormsg ("WriteFile: error %d writing drive %x sectors %lu..%lu: %s",
err, drv, sect, sect + num_sects - 1,
w32errtxt (err));
return false;
}
It's quite rare to want only GENERIC_WRITE. You most likely want GENERIC_READ|GENERIC_WRITE.
There is note in MSDN in documentation of CreateFile:
Direct access to the disk or to a volume is restricted. For more information, see "Changes to the file system and to the storage stack to restrict direct disk access and direct volume access in Windows Vista and in Windows Server 2008" in the Help and Support Knowledge Base at http://support.microsoft.com/kb/942448.
It refers to Vista/2008, but maybe apply to Win7 also.
I had a similar issue when porting from x86 to x64 code. You mention that you are passing null for your SECURITY_ATTRIBUTES parameter; I was getting access-denied errors myself using this approach until I actually began creating/passing this parameter.
I'm writing my first database application following a sample program the teacher given, but neither the sample, nor my own program can't connect to the database. (The JDBC sample program can, so the server should be OK).
I have these vars in the class declaration:
SQLHENV env;
SQLHDBC dbc;
SQLHSTMT stmt;
SQLRETURN ret;
Here's the constructor of my database handler class, that's where the connection should be made:
DBModule::DBModule(string server, string database)
{
this->server = server; //"localhost" is loaded into it
this->database = database; //"test" is loaded into it, of course it exists on the server
SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (void *) SQL_OV_ODBC3, 0);
SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);
command = "DRIVER={MySQL ODBC 3.51 Driver};SERVER="+this->server+";DATABASE="+this->database+";";
//command looks like this now:
//"DRIVER={MySQL ODBC 3.51 Driver};SERVER=localhost;DATABASE=test;"
ret = SQLDriverConnect(dbc, NULL, (SQLWCHAR *)command.c_str(), SQL_NTS, NULL, 0, NULL, SQL_DRIVER_COMPLETE);
if (!SQL_SUCCEEDED(ret)) {
err += CONNECT_DATABASE*DATABASE_UNREACHABLE;
good = false;
return;
} else {
good = true;
}
SQLAllocHandle(SQL_HANDLE_STMT, dbc, &stmt);
command = sysToStd(DBINIT);
SQLPrepare(stmt, (SQLWCHAR *)command.c_str(), SQL_NTS);
ret = SQLExecute(stmt);
}
The ret at SQLDriverConnect gets a -1 value.
I'm using the latest XAMPP as server with all the default settings (so i'm "root" and there is no password). I've tried adding UID=root to the connection string, but it did the same.
Thanks for any help.
You probably do not have MySQL ODBC drivers installed. JDBC works because you need not "install" them: they are some .jar files that can come with Java application. If you will use ODBC then install MySQL ODBC drivers, configure connection in ODBC Manager as System DSN, then from ODBC manager check if it connects to database (most ODBC drivers I know have "test connection" button).
When such test shows you "connected" or similar, then you can test if your application connects. Your connect sting looks like:
DRIVER={MySQL ODBC 3.51 Driver};SERVER=...;DATABASE....
so according to: http://www.connectionstrings.com/mysql#p30 it looks you are trying to use MySQL Connector/ODBC 3.51
Maybe database is not listening on dafult port?