I have a C++ code in which I am using sql loader using system(). When SQL Loader executes while running the code, I got below mentioned messages which I want to disable:
SQL*Loader: Release 10.2.0.1.0 - Production on Thu Mar 14 14:11:25 2013
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Commit point reached - logical record count 20
Commit point reached - logical record count 40
Commit point reached - logical record count 60
Commit point reached - logical record count 80
Remember that the system function uses the shell to execute the command. So you can use normal shell redirection:
system("/some/program > /dev/null");
You can use the silent=ALL option to suppress these messages:
system("/orahomepath/bin/sqlldr silent=ALL ...")
See also SQL*Loader Command-Line Reference:
As SQL*Loader executes, you also see feedback messages on the screen, for example:
Commit point reached - logical record count 20
You can suppress these messages by specifying SILENT with one or more values:
...
ALL - Implements all of the suppression values: HEADER, FEEDBACK, ERRORS, DISCARDS, and PARTITIONS.
Depending on the sql*ldr implementation, you might still end up with one or the other output - if you need complete silence, see the answer from #Joachim below.
Related
my project using QTcpSocket and the function setSocketDescriptor(). The code is very normal
QTcpSocket *socket = new QTcpSocket();
socket->setSocketDescriptor(this->m_socketDescriptor);
This coding worked fine most of the time until I ran a performance testing on Windows Server 2016, the crash occurred. I debugging with the crash dump, here is the log
0000004f`ad1ff4e0 : ucrtbase!abort+0x4e
00000000`6ed19790 : Qt5Core!qt_logging_to_console+0x15a
000001b7`79015508 : Qt5Core!QMessageLogger::fatal+0x6d
0000004f`ad1ff0f0 : Qt5Core!QEventDispatcherWin32::installMessageHook+0xc0
00000000`00000000 : Qt5Core!QEventDispatcherWin32::createInternalHwnd+0xf3
000001b7`785b0000 : Qt5Core!QEventDispatcherWin32::registerSocketNotifier+0x13e
000001b7`7ad57580 : Qt5Core!QSocketNotifier::QSocketNotifier+0xf9
00000000`00000001 : Qt5Network!QLocalSocket::socketDescriptor+0x4cf7
00000000`00000000 : Qt5Network!QAbstractSocket::setSocketDescriptor+0x256
In the stderr log, I see those logs
CreateWindow() for QEventDispatcherWin32 internal window failed (Not enough storage is available to process this command.)
Qt: INTERNAL ERROR: failed to install GetMessage hook: 8, Not enough storage is available to process this command.
Here is the function, where the code was stopped on the Qt codebase
void QEventDispatcherWin32::installMessageHook()
{
Q_D(QEventDispatcherWin32);
if (d->getMessageHook)
return;
// setup GetMessage hook needed to drive our posted events
d->getMessageHook = SetWindowsHookEx(WH_GETMESSAGE, (HOOKPROC) qt_GetMessageHook, NULL, GetCurrentThreadId());
if (Q_UNLIKELY(!d->getMessageHook)) {
int errorCode = GetLastError();
qFatal("Qt: INTERNAL ERROR: failed to install GetMessage hook: %d, %s",
errorCode, qPrintable(qt_error_string(errorCode)));
}
}
I did research and the error Not enough storage is available to process this command. maybe the OS (Windows) does not have enough resources to process this function (SetWindowsHookEx) and failed to create a hook, and then Qt fire a fatal signal, finally my app is killed.
I tested this on Windows Server 2019, the app is working fine, no crashes appear.
I just want to know more about the meaning of the error message (stderr) cause I don't really know what is "Not enough storage"? I think it is maybe the limit or bug of the Windows Server 2016? If yes, is there any way to overcome this issue on Windows Server 2016?
The error ‘Not enough storage is available to process this command’ usually occurs in Windows servers when the registry value is set incorrectly or after a recent reset or reinstallations, the configurations are not set correctly.
Below is verified procedure for this issue:
Click on Start > Run > regedit & press Enter
Find this key name HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\LanmanServer\Parameters
Locate IRPStackSize
If this value does not exist Right Click on Parameters key and Click on New > Dword Value and type in IRPStackSize under the name.
The name of the value must be exactly (combination of uppercase and lowercase letters) the same as what I have above.
Right Click on the IRPStackSize and click on Modify
Select Decimal enter a value higher than 15(Maximum Value is 50 decimal) and Click Ok
You can close the registry editor and restart your computer.
Reference
After researching for a few days I finally can configure the Windows Server 2016 setting (registry) to prevent the crash.
So basically it is a limitation of the OS itself, it is called desktop heap limitation.
https://learn.microsoft.com/en-us/troubleshoot/windows-server/performance/desktop-heap-limitation-out-of-memory
(The funny thing is the error message is Not enough storage is available to process this command but the real problem came to desktop heap limitation. )
So for the solution, flowing the steps in this link: https://learn.microsoft.com/en-us/troubleshoot/system-center/orchestrator/increase-maximum-number-concurrent-policy-instances
I increased the 3rd parameter of SharedSection to 2048 and it fix the issue.
Summary steps:
Desktop Heap for the non-interactive desktops is identified by the third parameter of the SharedSection= segment of the following registry value:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\SubSystems\Windows
The default data for this registry value will look something like the following:
%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,3072,512 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off MaxRequestThreads=16
The value to be entered into the Third Parameter of the SharedSection= segment should be based on the calculation of:
(number of desired concurrent policies) * 10 = (third parameter value)
Example: If it's desired to have 200 concurrent policy instances, then 200 * 10 = 2000, rounding up to a nice memory number gives you 2048as the third parameter resulting in the following update to be made to the registry value:
SharedSection=1024,3072,2048
I'm currently using the perf_event_open syscall (on Linux systems), and I try to understand a configuration parameter of this syscall which is given by the struct perf_event_attr structure.
It's about the read_format option.
Has anyone can see on the man page of this syscall, this parameter is related to the output of this call.
But I don't understand what every possible argument can do.
Especially these two possibilities:
PERF_FORMAT_TOTAL_TIME_ENABLED
PERF_FORMAT_TOTAL_TIME_RUNNING
Can anyone with that information give me a straight answer?
Ok.
I've looked a little further, and I think I have found an answer.
PERF_FORMAT_TOTAL_TIME_ENABLED: It seems that an "enabled time" refer to the difference between the time the event is no longer observed, and the time the event is registered as "to be observed".
PERF_FORMAT_TOTAL_TIME_RUNNING: It seems that an "running time" refer to the sum of the time the event is truly observed by the kernel. It's smaller or equal to PERF_FORMAT_TOTAL_TIME_ENABLED.
For example :
You tell to your kernel that you want to observe the X event at 1:13:05 PM. Your kernel create a "probe" on X, and start to record the activity.
Then, for an unknown reason, you tell to stop the record for the moment at 1:14:05 PM.
Then, you resume the record at 1:15:05 PM.
Finally, you stop the record at 1:15:35 PM.
You have 00:02:30 enabled time (1:15:35 PM - 1:13:05 PM = 00:02:30)
and 00:01:30 running time (1:14:05 PM - 1:13:05 PM + 1:15:35 PM - 1:15:05 PM = 00:01:30)
The read_format attribute can have both values using a mask. In C++, it looks like that :
event_configuration.read_format = PERF_FORMAT_TOTAL_TIME_ENABLED | PERF_FORMAT_TOTAL_TIME_RUNNING;
where event_configuration is an instance of struct perf_event_attr.
SBCL 1.3.9 produces the following error when I attempt to run the statistical profiler. Is start-profiling not exported?
* (in-package :cl-user)
* (require :sb-sprof)
* (sb-sprof:with-profiling (:report :flat) (bnb::solve))
; in: SB-SPROF:WITH-PROFILING (:REPORT :FLAT)
; (SB-SPROF:START-PROFILING :MAX-DEPTH 4611686018427387903 :THREADS
; (LIST SB-THREAD:*CURRENT-THREAD*))
;
; caught STYLE-WARNING:
; undefined function: SB-SPROF:START-PROFILING
;
; compilation unit finished
; Undefined function:
; SB-SPROF:START-PROFILING
; caught 1 STYLE-WARNING condition
debugger invoked on a UNDEFINED-FUNCTION in thread
#<THREAD "main thread" RUNNING {100292C913}>:
The function SB-SPROF:START-PROFILING is undefined.
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
It seems a problem with your distribution. please update to the last SBCL wich is
The most recent version is SBCL 1.3.18, released May 30, 2017
But if you take a look at the sources in github
git log -S start-profiling --source --all
commit 63f714af62d0ccdb9d4a793ab0245b036c3d8531 refs/tags/sbcl_1_0
Author: Juho Snellman <jsnell#iki.fi>
Date: Fri Nov 17 02:15:47 2006 +0000
0.9.18.58:
Further SB-SPROF improvements.
* Allocation profiling on gencgc. When the profiler is running in
allocation profiling mode, the gc will signal profiler ticks
when new allocation regions are opened.
* Add :LOOP keyword argument to WITH-PROFILING, to allow specifying
whether the body should be evaluated repeatedly until the maximum
sample count is reached.
* Improve merging of code-components with multiple debug-funs,
better handling of multiple functions with the same name
* More documentation
* Also update the stepper documentation
commit 554397512eea9d6e30067c5edc2def42006a5327 refs/tags/sbcl_0_8_12
Author: Christophe Rhodes <csr21#cam.ac.uk>
Date: Mon Jun 21 11:33:35 2004 +0000
0.8.11.20:
Add SB-SPROF contrib
That funcion was added a lot time ago, so please try the latest code and follow the manual
also if you inspect the code it has this mark
#-win32
So if you are using windows 32 bits this should not work
Is it possible to avoid the use of passwords when using the SAS Metadata Batch Export Tool?
I am building a feature in my STP web app (SAS 9.2, IWA, Kerberos) for auto-exporting metadata items. As per the documentation, the ExportPackage utility requires credentials either directly (-user and -password etc options) or via a connection profile (-profile).
Logging onto the application server as sassrv, the contents of my connection profile are as follows:
#Properties file updated on: Thu Mar 12 16:35:07 GMT 2015 !!!!! DO NOT EDIT !!!!!!!
#Thu Mar 12 16:35:07 GMT 2015
Name=SAS
port=8561
InternalAccount=false
host=DEV-SASMETA.somecompany.int
AppServer.Default=A5MNZZZZ.AR000666
AllowLocalPasswords=true
authenticationdomain=DefaultAuth
SingleSignOn=true
Running my code however results in the following:
44 +%put Batch tool located at: &platform_object_path;
Batch tool located at: C:\Program Files\SAS/SASPlatformObjectFramework/9.2
45 +filename inpipe pipe
46 + " ""&platform_object_path\ExportPackage"" -profile MyProfile
47 + -package 'C:\Temp\TestPackage.spk' -objects '/SomeFolder/ARCHIVE(Folder)' -includeDep -subprop";
48 +data _null_;
49 + infile inpipe;
50 + input; putlog _infile_;
51 +run;
NOTE: The infile INPIPE is:
Unnamed Pipe Access Device,
PROCESS="C:\Program Files\SAS/SASPlatformObjectFramework/9.2\ExportPackage" -profile MyProfile -package 'C:\Temp\TestPackage.spk' -objects '/SomeFolder/ARCHIVE(Folder)' -includeDep
-subprop,
RECFM=V,LRECL=256
The export process has failed. The native implementation module for the security package could not be found in the path.
For more information, view the export log file: C:\Users\sassrv\AppData\Roaming\SAS\Logs\Export_150427172003.log
NOTE: 2 records were read from the infile INPIPE.
The minimum record length was 112.
The maximum record length was 121.
The log file was empty.
Presumably my options here are limited to:
Requesting the user password from the front end
Using a system account in the connection profile
Using a system account in the -user & -password options
??
I verified this some time ago with SAS Technical Support; it's currently not possible.
I entered a SASWare Ballet item for it: https://communities.sas.com/t5/SASware-Ballot-Ideas/Add-Integrated-Windows-Authentication-IWA-support-to-Batch/idi-p/220474
please vote!
This was initially resolved by embedding a username / password in the profile, but now it works by taking the (modified) template profile below, and adding the host= / port= parameters dynamically at runtime (so can use the same profile in different environments).
IWA is now used to connect to the metadata server!
# This file is used by Release Management for connecting to the metadata server
# The host= and port= parameters are provided by the application at runtime
SingleSignOn=true
AllowLocalPasswords=true
InternalAccount=false
SecurityPackageList=Negotiate,NTLM
SecurityPackage=Negotiate
An important thing I discovered (not in SAS documentation) is that you can substitute a profile name with an absolute path to a profile (.swa file) in the ExportPackage commmand.
Edit (one year later):
As pointed out by #Stig Eide, the error does seem to relate to 32 vs 64 bit JREs. I also came across this issue in DI Studio today and solved it by copying the sspiauth.dll files as described here
I have a program that does some math in an SQL query. There are hundreds of thousands rows (some device measurements) in an SQLite table, and using this query, the application breaks these measurements into groups of, for example, 10000 records, and calculates the average for each group. Then it returns the average value for each of these groups.
The query looks like this:
SELECT strftime('%s',Min(Stamp)) AS DateTimeStamp,
AVG(P) AS MeasuredValue,
((100 * (strftime('%s', [Stamp]) - 1334580095)) /
(1336504574 - 1334580095)) AS SubIntervalNumber
FROM LogValues
WHERE ((DeviceID=1) AND (Stamp >= datetime(1334580095, 'unixepoch')) AND
(Stamp <= datetime(1336504574, 'unixepoch')))
GROUP BY ((100 * (strftime('%s', [Stamp]) - 1334580095)) /
(1336504574 - 1334580095)) ORDER BY MIN(Stamp)
The numbers in this request are substituted by my application with some values.
I don't know if i can optimize this request more (if anyone could help me to do so, i'd really appreciate)..
This SQL query can be executed using an SQLite command line shell (sqlite3.exe). On my Intel Core i5 machine it takes 4 seconds to complete (there are 100000 records in the database that are being processed).
Now, if i write a C program, using sqlite.h C interface, I am waiting for 14 seconds for exactly the same query to complete. This C program "waits" during these 14 seconds on the first sqlite3_step() function call (any following sqlite3_step() calls are executed immediately).
From the Sqlite download page I have downloaded SQLite command line shell's source code and build it using Visual Studio 2008. I ran it and executed the query. Again 14 seconds.
So why does a prebuilt, downloaded from the sqlite website, command line tool takes only 4 seconds, while the same tool, built by me, takes 4 times longer time to execute?
I am running Windows 64 bit. The prebuilt tool is an x86 process. It also does not seem to be multicore optimized - in a Task Manager, during query execution, I can see only one core busy, for both built-by-mine and prebuilt SQLite shells.
Any way I could make my C program execute this query as fast as the prebuilt command line tool does it?
Thanks!