Build time of XPages application containing several JARs, Java sources and ~50 XP/CC elements takes about minute to build on server via WAN. I have replicated application to local, build time dropped to ~10s.
Since few days ago build of local application is extremely slow, about 2-5 minutes. After some experiments there is workaround: to disable TCP port in location document - it drops build times to just few seconds. Even tho it works, it does not help much - testing requires user to be authenticated, so I need to replicate design changes to remote or local server - and that means to change location (online/offline) every time.
UPDATE 2013-04-04: I have duplicated my current location document and removed home and directory servers. To my surprise, with this location build times went back to few seconds - with TCP port enabled so replication is possible. Bigger surprise was the fact, that returning home/directory servers back to new location did not reproduce the problem - in fact they do not affect performance. I know it because I have renamed current location document and everything went to normal. From my understanding, "something" in client configuration was connected to location name. Thanks to Simon's tips I will investigate further.
The question is still open: I am looking for some (eclipse) preference controlling this behavior - unintended communication with server during build of local application.
Solution:
Teamstudio CIAO hooks into designer and checks for every update of design element. Seems to be lack of code optimization to me: it checks whether currently built design element (every single one, one by one) should be controlled in CIAO config database.
This explains why the problem was solved by renaming of location document. I was disappointed yesterday, when performance problems started again. Fortunately, I recalled CIAO setup to that location document about that time. CIAO uses teamstudio.ini file in DATA directory to configure what CIAO configuration database is used for every location document. Look for entry:
CIAOConfigDb[location name]=server name;CIAO\CIAOConfig.nsf
For development on local replicas with connection to server (for replication or local server), use location document with CIAO disabled.
This works only with property ForceConfigLocation=0.
Not a solution (yet!), but may help in the investigation. I'll update further if you post results later.
Debug instructions.
Add the following to the shortcut that launches the Designer client.
-RPARAMS -console -debug -separateSysLogFiles -consoleLog
Start the designer client. This will also open up the OSGi console.
Reproduce the issue. While it is still in progress in the OSGi console type the following:
dump threads
Do this three times, with a small amount of time between completion of each dump. Once done open the three heap dumps (in the IBM_TECHNICAL_SUPPORT folder) in the Heap Dump Analyser.
It will show you what threads are consistent through all three dumps. Take a look at those and look for package names/calls which may appear to be a functional area. Once you have that then you can try adding the debug for the related class.
For example: Let's say you notice "com.ibm.designer.domino.ui.commons." in the thread, then you would edit the rcpinstall.properties file. It will be in:
<Notes Install>\Data\workspace\.config\rcpinstall.properties
and you would add (start with FINE, then FINEST if nothing):
com.ibm.designer.domino.ui.commons.level=FINE
Now when you restart the designer client it will generate debug output in the workspace\logs folder for that package. You need to then go through the trace logs looking for the time when the delay occurred and see if it makes any references to related design elements.
Other open applications may get built at the same time (which looks like a bug top me). Be sure to close all other applications and the server based replica. Open applications have their icon showing in the application list and they stay open even if you close and reopen the Designer. In Designer 9 right click application and select "Close Application". In 8.5 you need to use Package Exprorer for closing.
Another good way is to use Working Sets. Only applications in open Working Set will be built (AFAIK). Have a Working Set with this one app only (and the app only in this Working Set).
update 1
If these don't help I would delete/rename bookmark.nsf, Cache.NDK and desktop8.ndk. Then open just this one app and see what happens.
update 2
Check that there are no referenced projects. Right click the application and select "Project Properties". From there "Project Referencies" and make sure no check boxes are checked.
update 3
Based on your update I would check the item names starting with $ in location document. Sometimes there are saved IP addresses etc. which could cause this problem. All those items can be removed.
If possible (and if You are not using it yet) try to use version 9 of the Domino designer (You do not have to use Domino 9 to do that - it works fine with Domino 8.5.3).
For our projects build times went down to only few seconds from few minutes. I guess that they finally noticed at IBM that the build process used to heavily relay on connection to server and done something with it.
With new designer You don't event have to replicate to local. You can directly work on Your local server.
Related
I have followed this tutorial for developing an operating system. I am using Windows 10 as my host sytem and used wsl for compiling. But my VM fails as soon as I enable interrupts.
This is the log file of the VM that is output, but I cannot understand it. I am pretty naive with VirtualBox. Can someone explain any possible error you see?
Here is the code of the Os. I just have changed the structure I believe. Rest code in execution point of view is same as shown in video series.
That is a lot of log to scroll through and it's hard to be sure on the face of it that just looking at that would be able to tell us what about your startup code (not visible to us as part of the question) would trigger it. However, I can speak to some general strategies about approaching a log file like this.
We can see some general state transitions in there. The log ends with:
00:00:15.712045 Changing the VM state from 'DESTROYING' to 'TERMINATED'
So I can go back through and look at where the first instance of DESTROYING showed up, which was:
00:00:15.698320 Changing the VM state from 'POWERING_OFF' to 'OFF'
00:00:15.701802 Changing the VM state from 'OFF' to 'DESTROYING'
Following the same process backwards to POWERING_OFF, I see:
00:00:08.577363 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
00:00:14.342287 ERROR [COM]: aRC=VBOX_E_INVALID_VM_STATE (0x80bb0002) aIID={872da645-4a9b-1727-bee2-5585105b9eed} aComponent={ConsoleWrap} aText={Invalid machine state GuruMeditation when checking if the guest entered the ACPI mode)}, preserve=false aResultDetail=0
00:00:15.643579 GUI: Request for close-action to power VM off.
00:00:15.643599 GUI: Passing request to power VM off from machine-logic to UI session.
00:00:15.643606 GUI: Powering VM down on UI session power off request...
00:00:15.644257 Console: Machine state changed to 'Stopping'
00:00:15.644763 Console::powerDown(): A request to power off the VM has been issued (mMachineState=Stopping, InUninit=0)
00:00:15.645075 Changing the VM state from 'GURU_MEDITATION' to 'POWERING_OFF'
That error line at the top of that block may point to something searchable that would turn up other instances of people having the same or a similar problem. If you scroll up a bit, you can also see that something VGA-related was happening right before the error, which may help narrow it down if it's directly related to the error, or may be another step to backtrack through on the way to the real issue.
I'm having problem making a permanent calibration in my embedded solution. I'm developing a Qt-based app for a Embedded Linux environment with touch screen. For this last part, I use tslib (configured by previous developers).
In what comes to simply calibrate the touch screen, everything is fine: ts_calibrate runs and creates the pointercal file correctly. If after calling ts_calibrate I run my Qt app (or ts_test), I can notice that the calibration is successful.
The problem is that the calibration results only works for 1 opening of my app: I calibrate with ts_calibrate, run my app, close it and if I run my app again, the screen is one again non-calibrated.
Now obviously I don't want to have to call the calibration each time my app is closed and reopened. The question is: how to make the calibration results become permanent? (that is, till another calibration is made)
Extra info:
I did some research on the web and I found this SO thread telling about a way to handle this problem using QWSServer. At first I disliked this solution since it depends on the Qt framework to do the job (I was expecting a more general, "C++ solution" (or a call to a script, whatever)). But I implemented it and it worked - but only in a specific case, namely, if I calibrate, open my software, close it and reopen it, then the calibration is maintained. But the problem nevertheless persists if I shut down the hardware completely, turn it on and run my app without a call to ts_calibrate (reloading the Linux kernel in the process); so this show to be only a partial solution and, therefore, not acceptable.
Trying to find the source of the problem, I created a copy of the pointercal file just after calibration and another copy of it after shutting down and turning up my hardware (and confirming that the calibration was over) and I noticed that the file was changed in the middle despite no call to the ts_calibrate or similar app was made:
After calibration:
55438 118 -1920736 -543 -36058 34531168 65536 800 480
After hardware shutdown:
-55040 1280 2526720 -288 35040 -34398240 -62768
The terminal log for the linux boot (tftp; bootm command) don't mention pointercal or a relevant calibration process.
Edit
I recently learned that the pointercal file located inside /etc/ is changing between sections because that entire folder is made new when the hardware is restarted. So what is essentially happening is that Tslib is going after a file that is constantly reset to default each time the hardware is restarted, and what I need to do is to configure Tslib not to look there, but to a more secure folder (in my case, the SD Card). The new question now is: how to do that? I know I have to configure the tslib.sh file making the TSLIB_CALIBFILE variable point to the new location of pointerscal, but tslib.shis itself inside /etc/, being itself temporary.
You have to change TSLIB_CALIBFILE in the image loaded via tftp.
That should do it, since you just have to change that once.
I created a logging module which logs messages to a mysql db, the current code is located here:
https://github.com/amiadogroup/mod_log_chat_mysql5/blob/master/src/mod_log_chat_mysql5.erl
The Problem with the current code is, that sometimes the connection gets closed and as a result, the module doesn't work anymore.
As you see in the code, I store the DBRef in an ets table, which is not really the good way to go.
I asked the erlang mailinglist about this and they suggested me to do the DB Connection as an own child process of the module. This would enable the module to gracefully restart the connection upon closing of the connection.
Now my question is: how can I implement this child process with gen_server and/or gen_mod?
Do I need to create two files or can I do it within the same file?
Is there any example somewhere on how I could achieve that?
Edit: As you can see in the linked github repo, I updated the code and it works now, weeha!
Looking at the mod_Archive code helped me a lot, although I didn't decide to upgrade my ejabberd version.
I ran into another, but related problem now. In the code you see that I do an initial query with "SET NAMES UTF8" to prevent garbling of messages. It seems that this isn't done again if the gen_server does a reconnect. Is there any hook I can call upon reconnect so that the UTF8 query is done everytime?
Edit#2:
Now I switched to Emysql (https://github.com/Eonblast/Emysql) and it works out of the box by specifying the encoding directly on connect.
Code is on github.
Thanks for your help,
Michael
I suggest you look into general Erlang/OTP principles (gen_server, supervisor, etc).
ejabberd is relying on this standard Erlang architecture pattern.
Regarding your comment on database, ejabberd has its own way on managing database and passing queries to MySQL for example. You should as well look into it.
In your source code you are only applying the gen_mod behaviour, if you do wish to have a gen_server you can do it in the same module, if you define the gen_server behaviour has well.
A good example would be the ejabberd module mod_archive, which implements both behaviours.
Edit: I never really worked "directly" with mysql on erlang. But through the ejabberd methods I find it pretty "easy"(you will have to make a few setup, but rather easy). You have the method
ejabberd_odbc:sql_query_t(Query)
And has an example you can find it on the module mod_archive_odbc.
To use that method(and the last module) I haved downloaded the mysql native driver and put the beams created from the driver in ejabberd ebin dir (you can put it anywhere has long is on the erlang path).
A a soft link to the ejabberd ebin is my favorite:
ln -s <diryouhavethedriver>/ebin/*.beam /usr/lib/ejabberd/ebin/
and do a few configurations on you ejabberd.cfg. This process is described on this page on process one. Notice that the full steps are to make mysql the full database of ejabberd. You may not want that, so you must jump a few steps.
Hope this help.
I'm trying to test the happy-path for a piece of code which takes a long time to respond, and then begins writing a file to the response output stream, which prompts a download dialog in browsers.
The problem is that this process has failed in the past, throwing an exception after this long amount of work. Is there a way in selenium to wait-for-download or equivalent?
I could throw in a Thread.sleep, but that would be inaccurate and unnecessarily slow down the test run.
What should I do, here?
I had the same problem. I invented something to solve the problem. A tempt file is created by Python with '.part' extension. So, if still we have the temp, python can wait for 10 second and check again if the file is downloaded or not yet.
while True:
if os.path.isfile('ts.csv.part'):
sleep(10)
elif os.path.isfile('ts.csv'):
break
else:
sleep(10)
driver.close()
So you have two problems here:
You need to cause the browser to download the file
You need to measure when the downloaded file is complete
Neither problemc an be directly solved by Selenium (yet - 2.0 may help), but both are solvable problems. The first problem can be solved by GUI automation toolkits, such as AutoIT. But they can also be solved by simply sending an automated keypress at the OS level that simulates the enter key (works for Firefox, a little harder on some versions of Chrome and Safari). If you're using Java, you can use Robot to do that. Other languages have similar toolkits to do such a thing.
The second issue is probably best solved with some sort of proxy solution. For example, if your browser was configured to go through a proxy and that proxy had an API, you could query the proxy with that API to ask when network activity had ended.
That's what we do at http://browsermob.com, which is a a startup I founded that uses Selenium to do load testing. We've released some of the proxy code as open source, available at http://browsermob.com/tools.
But two problems still persist:
You need to configure the browser to use the proxy. In Selenium 2 this is easier, but it's possible to do it with Selenium 1 as well. The key is just making sure that your browser launcher brings up the browser with the right profile/settings.
There currently is no API for BrowserMob proxy to tell you when network traffic has stopped! This is a big hole in the concept of the project that I want to fix as soon as I get the time. However, if you're keen to help out, join the Google Group and I can definitely point you in the right direction.
Hope that helps you identify your various options. Best of luck!
This is Chrome-testing-only solution for controlling the downloads with javascript..
Using WebDriver (Selenium2) it can be done within Chrome's chrome:// which is HTML/CSS/Javascript:
driver.get( "chrome://downloads/" );
waitElement( By.CssSelector("#downloads-summary-text") );
// next javascript snippet cancels the last/current download
// if your test ends in file attachment downloading
// you'll very likely need this if you more re-instantiated tests left
((JavascriptExecutor)driver).executeScript("downloads.downloads_[0].cancel_();");
There are other Download.prototype.functions in "chrome://downloads/downloads.js"
This suites you if you just need to test some info note eg. caused by file attachment starting activity, and not the file itself.
Naturally you need to control step 1. - mentioned by Patrick above - and by this you control step 2. FOR THE TEST, not for the functionality of actual file download completion / cancel.
See also : Javascript: Cancel/Stop Image Requests which is relating to Browser stopping.
This falls under the "things that can't be automated" category. Selenium is built with JavaScipt and due to JavaScript sandbox restrictions it can't access downloads.
Selenium 2 might be able to do this once Alerts/Prompts have been implemented but that this won't happen for the next little while yet.
If you want to check for the download dialog, try with AutoIt. I use that for uploading and downloading the files. Using AutoIt with Se RC is easier.
def file_downloaded?(file)
while File.file?(file) == false
p "File downloading in progress..."
sleep 1
end
end
*Ruby Syntax
Trying to honor a feature request from our customers, I'd like that my application, when Internet is available, check on our website if a new version is available.
The problem is that I have no idea about what have to be done on the server side.
I can imagine that my application (developped in C++ using Qt) has to send a request (HTTP ?) to the server, but what is going to respond to this request ? In order to go through firewalls, I guess I'll have to use port 80 ? Is this correct ?
Or, for such a feature, do I have to ask our network admin to open a specific port number through which I'll communicate ?
#pilif : thanks for your detailed answer. There is still something which is unclear for me :
like
http://www.example.com/update?version=1.2.4
Then you can return what ever you want, probably also the download-URL of the installer of the new version.
How do I return something ? Will it be a php or asp page (I know nothing about PHP nor ASP, I have to confess) ? How can I decode the ?version=1.2.4 part in order to return something accordingly ?
I would absolutely recommend to just do a plain HTTP request to your website. Everything else is bound to fail.
I'd make a HTTP GET request to a certain page on your site containing the version of the local application.
like
http://www.example.com/update?version=1.2.4
Then you can return what ever you want, probably also the download-URL of the installer of the new version.
Why not just put a static file with the latest version to the server and let the client decide? Because you may want (or need) to have control over the process. Maybe 1.2 won't be compatible with the server in the future, so you want the server to force the update to 1.3, but the update from 1.2.4 to 1.2.6 could be uncritical, so you might want to present the client with an optional update.
Or you want to have a breakdown over the installed base.
Or whatever. Usually, I've learned it's best to keep as much intelligence on the server, because the server is what you have ultimate control over.
Speaking here with a bit of experience in the field, here's a small preview of what can (and will - trust me) go wrong:
Your Application will be prevented from making HTTP-Requests by the various Personal Firewall applications out there.
A considerable percentage of users won't have the needed permissions to actually get the update process going.
Even if your users have allowed the old version past their personal firewall, said tool will complain because the .EXE has changed and will recommend the user not to allow the new exe to connect (users usually comply with the wishes of their security tool here).
In managed environments, you'll be shot and hanged (not necessarily in that order) for loading executable content from the web and then actually executing it.
So to keep the damage as low as possible,
fail silently when you can't connect to the update server
before updating, make sure that you have write-permission to the install directory and warn the user if you do not, or just don't update at all.
Provide a way for administrators to turn the auto-update off.
It's no fun to do what you are about to do - especially when you deal with non technically inclined users as I had to numerous times.
Pilif answer was good, and I have lots of experience with this too, but I'd like to add something more:
Remember that if you start yourapp.exe, then the "updater" will try to overwrite yourapp.exe with the newest version. Depending upon your operating system and programming environment (you've mentioned C++/QT, I have no experience with those), you will not be able to overwrite yourapp.exe because it will be in use.
What I have done is create a launcher. I have a MyAppLauncher.exe that uses a config file (xml, very simple) to launch the "real exe". Should a new version exist, the Launcher can update the "real exe" because it's not in use, and then relaunch the new version.
Just keep that in mind and you'll be safe.
Martin,
you are absolutely right of course. But I would deliver the launcher with the installer. Or just download the installer, launch it and quit myself as soon as possible. The reason is bugs in the launcher. You would never, ever, want to be dependent on a component you cannot update (or forget to include in the initial drop).
So the payload I distribute with the updating process of my application is just the standard installer, but devoid of any significant UI. Once the client has checked that the installer has a chance of running successfully and once it has downloaded the updater, it runs that and quits itself.
The updater than runs, installs its payload into the original installation directory and restarts the (hopefully updated) application.
Still: The process is hairy and you better think twice before implementing an Auto Update functionality on the Windows Platform when your application has a wide focus of usage.
in php, the thing is easy:
<?php
if (version_compare($_GET['version'], "1.4.0") < 0){
echo "http://www.example.com/update.exe";
}else{
echo "no update";
}
?>
if course you could extend this so the currently available version isn't hard-coded inside the script, but this is just about illustrating the point.
In your application you would have this pseudo code:
result = makeHTTPRequest("http://www.example.com/update?version=" + getExeVersion());
if result != "no update" then
updater = downloadUpdater(result);
ShellExecute(updater);
ExitApplication;
end;
Feel free to extend the "protocol" by specifying something the PHP script could return to tell the client whether it's an important, mandatory update or not.
Or you can add some text to display to the user - maybe containing some information about what's changed.
Your possibilities are quite limitless.
My Qt app just uses QHttp to read tiny XML file off my website that contains the latest version number. If this is greater than the current version number it gives the option to go to the download page. Very simple. Works fine.
I would agree with #Martin and #Pilif's answer, but add;
Consider allowing your end-users to decide if they want to actually install the update there and then, or delay the installation of the update until they've finished using the program.
I don't know the purpose/function of your app but many applications are launched when the user needs to do something specific there and then - nothing more annoying than launching an app and then being told it's found a new version, and you having to wait for it to download, shut down the app and relaunch itself. If your program has other resources that might be updated (reference files, databases etc) the problem gets worse.
We had an EPOS system running in about 400 shops, and initially we thought it would be great to have the program spot updates and download them (using a file containing a version number very similar to the suggestions you have above)... great idea. Until all of the shops started up their systems at around the same time (8:45-8:50am), and our server was hit serving a 20+Mb download to 400 remote servers, which would then update the local software and cause a restart. Chaos - with nobody able to trade for about 10 minutes.
Needless to say that this caused us to subsequently turn off the 'check for updates' feature and redesign it to allow the shops to 'delay' the update until later in the day. :-)
EDIT: And if anyone from ADOBE is reading - for god's sake why does the damn acrobat reader insist on trying to download updates and crap when I just want to fire-it-up to read a document? Isn't it slow enough at starting, and bloated enough, as it is, without wasting a further 20-30 seconds of my life looking for updates every time I want to read a PDF?
DONT THEY USE THEIR OWN SOFTWARE??!!! :-)
On the server you could just have a simple file "latestversion.txt" which contains the version number (and maybe download URL) of the latest version. The client then just needs to read this file using a simple HTTP request (yes, to port 80) to retrieve http://your.web.site/latestversion.txt, which you can then parse to get the version number. This way you don't need any fancy server code --- you just need to add a simple file to your existing website.
if you keep your files in the update directory on example.com, this PHP script should download them for you given the request previously mentioned. (your update would be yourprogram.1.2.4.exe
$version = $_GET['version'];
$filename = "yourprogram" . $version . ".exe";
$filesize = filesize($filename);
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: post-check=0, pre-check=0");
header("Content-type: application-download");
header('Content-Length: ' . $filesize);
header('Content-Disposition: attachment; filename="' . basename($filename).'"');
header("Content-Transfer-Encoding: binary");
This makes your web browser think it's downloading an application.
The simplest way to make this happen is to fire an HTTP request using a library like libcurl and make it download an ini or xml file which contains the online version and where a new version would be available online.
After parsing the xml file you can determine if a new version is needed and download the new version with libcurl and install it.
Just put an (XML) file on your server with the version number of the latest version, and a URL to the download the new version from. Your application can then request the XML file, look if the version differs from its own, and take action accordingly.
I think that simple XML file on the server would be sufficient for version checking only purposes.
You would need then only an ftp account on your server and build system that is able to send a file via ftp after it has built a new version. That build system could even put installation files/zip on your website directly!
If you want to keep it really basic, simply upload a version.txt to a webserver, that contains an integer version number. Download that check against the latest version.txt you downloaded and then just download the msi or setup package and run it.
More advanced versions would be to use rss, xml or similar. It would be best to use a third-party library to parse the rss and you could include information that is displayed to your user about changes if you wish to do so.
Basically you just need simple download functionality.
Both these solutions will only require you to access port 80 outgoing from the client side. This should normally not require any changes to firewalls or networking (on the client side) and you simply need to have a internet facing web server (web hosting, colocation or your own server - all would work here).
There are a couple of commercial auto-update solutions available. I'll leave the recommendations for those to others answerers, because I only have experience on the .net side with Click-Once and Updater Application Block (the latter is not continued any more).