I've looked at all the suggestions and I am trying to set the max content size value to a small (700) just to prove I can change it. Jetty 9.4.41.
When I send a POST like:
0000: POST /data1/MaxRequestSize/MyTryURL01 HTTP/1.1\r\n
0030: Accept: application/xml,application/atom+xml\r\n
005E: User-Agent: xxxxxxxxxxx\r\n
0077: Content-Type: application/atom+xml; charset=utf-8\r\n
00AA: Host: localhost:9090\r\n
00C0: Connection: keep-alive\r\n
00D8: Content-Length: 808\r\n
00ED: \r\n
...
I get no error (i.e 808 > 700).
Extracting my server startup code:
HandlerCollection handlers = new HandlerCollection();
System.setProperty( "org.eclipse.jetty.server.Request.maxFormContentSize", "700" ); /**1**/
server = new Server();
server.setHandler( handlers );
server.setAttribute( "org.eclipse.jetty.server.Request.maxFormContentSize", 700 ); /**2**/
// Setting up connection
ServerConnector serverConn = null;
HttpConfiguration httpConfig = new HttpConfiguration();
httpConfig.setSendServerVersion( false );
httpConfig.setSendXPoweredBy( false );
HttpConnectionFactory hcf = new HttpConnectionFactory( httpConfig );
serverConn = new ServerConnector( server, hcf );
server.addConnector( serverConn );
serverConn.setPort( httpPort );
serverConn.setName( Constants.CONF_HTTP_CONNECTOR );
serverConn.setReuseAddress( true );
// Initializing servlets
contextHandlers = new ContextHandlerCollection();
ServletHolder[] holders = new ServletHolder[ 1 ];
ServletHolder holder = new ServletHolder( myServlet );
holder.setInitParameter( ... );
ServletContextHandler handler = new ServletContextHandler();
handler.setMaxFormContentSize( 700 ); /**3**/
handler.setContextPath( servicePath );
handler.setAllowNullPathInfo( true );
connectors = new String[ 2 ];
connectors[0] = "#" + Constants.CONF_HTTP_CONNECTOR;
connectors[1] = "#" + Constants.CONF_ADMIN_CONNECTOR;
handler.setVirtualHosts( connectors );
handler.setDisplayName( "foo" );
handler.addServlet( holder, "/*" );
contextHandlers.addHandler( handler );
// Launching
handlers.addHandler( contextHandlers );
server.start();
Any suggestion appreciated.
Jetty only enforces a max limit for application/x-www-form-urlencoded. My tests did not use that particular encoding.
Related
My script is working so far to open a remote FTP connection, change directory, and download a file. My last two steps would be to delete the remove file once it's fully downloaded and then close the connection. ACF documentation (and cfdocs) seems to have very little information on this. Here's what I have so far:
ftpConnection = ftpService.open(
action = 'open',
connection = variables.ftpConnectionName,
server = variables.ftpServerName,
username = '***************',
password = '***************',
secure='true');
if( ftpConnection.getPrefix().succeeded ){
fileList = ftpService.listdir(directory = variables.ftpPath, connection= variables.ftpConnectionName, name='pendingFiles', stopOnError='true').getResult();
if( fileList.recordCount ){
changeFtpConnectionDir = ftpService.changeDir(
connection = variables.ftpConnectionName,
directory = variables.ftpPath);
getFtpConnection = ftpService.getFile(
connection = variables.ftpConnectionName,
remoteFile = fileList.name,
localFile = local.localPath & fileList.name,
failIfExists = false,
timeout = 3000
);
deleteRemoteFile = ftpService.remove(
connection = variables.ftpConnectionName,
remoteFile = fileList.name
};
closeFtp = ftpService.close(
connection = variables.ftpConnectionName
);
};
};
Error is thrown on the remoteFile = fileList.name. Since I already changed directory I don't think I need to put the full path here.
I put the entire script up since there doesn't seem to be many resources out there about using the newer ftpServer() functions.
D'oh - my issue was a typo:
deleteRemoteFile = ftpService.remove(
connection = variables.ftpConnectionName,
remoteFile = fileList.name
);// had } instead of )
I'll still leave this up as a resource for ftpService()
I'm trying to configure https://github.com/Atlantic18/DoctrineExtensions
with the instruction https://github.com/l3pp4rd/DoctrineExtensions/blob/master/doc/annotations.md#em-setup
But I get an error:
The annotation "#Doctrine\ORM\Mapping\MappedSuperclass" in class Gedmo\Translatable\Entity\MappedSuperclass\AbstractTranslation does not exist, or could not be auto-loaded\MappedSuperclass" in class Gedmo\Translatable\Entity\MappedSuperclass\AbstractTranslation does not exist, or could not be auto-loaded.
My configs
define('DB_CONNECTION', "host= ....");
$file_text_lib = __DIR__ . "/../../vendor/autoload.php";
require_once($file_text_lib);
// bootstrap.php
use Doctrine\ORM\Tools\Setup;
use Doctrine\ORM\EntityManager;
use Symfony\Component\Console\Helper\HelperSet;
use Symfony\Component\Console\Helper\QuestionHelper;
use Symfony\Component\Console\Application;
use Doctrine\ORM\Version;
//use Doctrine\ORM\EntityManagerInterface;
use Doctrine\DBAL\Tools\Console\Helper\ConnectionHelper;
use Doctrine\ORM\Tools\Console\Helper\EntityManagerHelper;
define(
"PARSE_CONNECTION_STRING_REGEXP",
"/(\s*host\s*=\s*(?P<host>[^\s]+)|\s*port\s*=\s*(?P<port>[\d]+)|\s*dbname\s*=\s*(?P<dbname>[^\s]+)|\s*user\s*=\s*(?P<user>[^\s]+)|\s*password\s*=\s*(?P<password>[^\s]+)\s*)*/"
);
$isDevMode = true;
// globally used cache driver, in production use APC or memcached
$cache = new Doctrine\Common\Cache\ArrayCache;
// standard annotation reader
$annotationReader = new Doctrine\Common\Annotations\AnnotationReader;
$cachedAnnotationReader = new Doctrine\Common\Annotations\CachedReader(
$annotationReader, // use reader
$cache // and a cache driver
);
// create a driver chain for metadata reading
$driverChain = new Doctrine\Common\Persistence\Mapping\Driver\MappingDriverChain();
// load superclass metadata mapping only, into driver chain
// also registers Gedmo annotations.NOTE: you can personalize it
Gedmo\DoctrineExtensions::registerAbstractMappingIntoDriverChainORM(
$driverChain, // our metadata driver chain, to hook into
$cachedAnnotationReader // our cached annotation reader
);
// now we want to register our application entities,
// for that we need another metadata driver used for Entity namespace
$ymlDriver = new Doctrine\ORM\Mapping\Driver\YamlDriver(array(__DIR__."/YAMLMetaConfiguration"));
$driverChain->addDriver($ymlDriver, 'SemanticPersistence\\Entities');
// general ORM configuration
$config = new Doctrine\ORM\Configuration;
$config->setProxyDir(__DIR__ . "/../Proxies");
$config->setProxyNamespace('SemanticPersistence\\Proxies');
$config->setAutoGenerateProxyClasses($isDevMode); // this can be based on production config.
// register metadata driver
$config->setMetadataDriverImpl($driverChain);
// use our already initialized cache driver
$config->setMetadataCacheImpl($cache);
$config->setQueryCacheImpl($cache);
$evm = new Doctrine\Common\EventManager();
// timestampable
$timestampableListener = new Gedmo\Timestampable\TimestampableListener;
$timestampableListener->setAnnotationReader($cachedAnnotationReader);
$evm->addEventSubscriber($timestampableListener);
//$config = Setup::createAnnotationMetadataConfiguration(array(__DIR__ . "/Entities"), $isDevMode, null, null, false);
// or if you prefer yaml or XML
//$config = Setup::createXMLMetadataConfiguration(array(__DIR__."/config/xml"), $isDevMode);
/*
$config = Setup::createYAMLMetadataConfiguration(array(__DIR__."/YAMLMetaConfiguration"), $isDevMode);
$config->setAutoGenerateProxyClasses(true);
$config->setProxyDir(__DIR__ . "/../Proxies");
$config->setProxyNamespace('SemanticPersistence\\Proxies');
*/
preg_match(
PARSE_CONNECTION_STRING_REGEXP,
DB_CONNECTION,
$matches
);
$dbParams = array(
'driver' => 'pdo_pgsql',
'user' => $matches['user'],
'password' => $matches['password'],
'host' => $matches['host'],
'dbname' => $matches['dbname'],
'charset' => 'UTF-8'
);
if (!empty($matches['port'])) {
$dbParams['port'] = $matches['port'];
}
$entityManager = Doctrine\ORM\EntityManager::create($dbParams, $config, $evm);
$helperSet = new HelperSet(array(
'db' => new ConnectionHelper($entityManager->getConnection()),
'em' => new EntityManagerHelper($entityManager),
'dialog' => new QuestionHelper(),
));
$commands = array(
// DBAL Commands
new \Doctrine\DBAL\Tools\Console\Command\RunSqlCommand(),
new \Doctrine\DBAL\Tools\Console\Command\ImportCommand(),
// ORM Commands
new \Doctrine\ORM\Tools\Console\Command\ClearCache\MetadataCommand(),
new \Doctrine\ORM\Tools\Console\Command\ClearCache\ResultCommand(),
new \Doctrine\ORM\Tools\Console\Command\ClearCache\QueryCommand(),
new \Doctrine\ORM\Tools\Console\Command\SchemaTool\CreateCommand(),
new \Doctrine\ORM\Tools\Console\Command\SchemaTool\UpdateCommand(),
new \Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand(),
new \Doctrine\ORM\Tools\Console\Command\EnsureProductionSettingsCommand(),
new \Doctrine\ORM\Tools\Console\Command\ConvertDoctrine1SchemaCommand(),
new \Doctrine\ORM\Tools\Console\Command\GenerateRepositoriesCommand(),
new \Doctrine\ORM\Tools\Console\Command\GenerateEntitiesCommand(),
new \Doctrine\ORM\Tools\Console\Command\GenerateProxiesCommand(),
new \Doctrine\ORM\Tools\Console\Command\ConvertMappingCommand(),
new \Doctrine\ORM\Tools\Console\Command\RunDqlCommand(),
new \Doctrine\ORM\Tools\Console\Command\ValidateSchemaCommand(),
new \Doctrine\ORM\Tools\Console\Command\InfoCommand(),
//new \Doctrine\ORM\Tools\Console\Command\MappingDescribeCommand(),
);
$cli = new Application('Doctrine Command Line Interface', Version::VERSION);
$cli->setCatchExceptions(true);
$cli->setHelperSet($helperSet);
$cli->addCommands($commands);
$cli->run();
What am I doing wrong?
Forgot to ensure standard doctrine annotations are registered
Doctrine\Common\Annotations\AnnotationRegistry::registerFile(
__DIR__.'/../vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/Driver/DoctrineAnnotations.php'
);
ะก++, Embarcadero RAD Studio XE2
I need connect to a ::6100 with TIdTCPClient through http-proxy. So I wrote this code:
m_pClient = new TIdTCPClient( NULL );
m_pClient->Host = m_sServerAddress.c_str();
m_pClient->Port = StrToInt( m_sServerPort.c_str() );
m_pClient->ConnectTimeout = 5000;
m_pClient->ReadTimeout = 5000;
if ( m_bUseProxy == true )
{
m_pIdIOHandlerStack = new TIdIOHandlerStack( NULL );
m_pIdIOHandlerStack->TransparentProxy = new TIdConnectThroughHttpProxy( m_pIdIOHandlerStack );
m_pIdIOHandlerStack->TransparentProxy->Host = m_sProxyHost;
m_pIdIOHandlerStack->TransparentProxy->Port = m_iProxyPort;
m_pIdIOHandlerStack->TransparentProxy->Enabled = True;
m_pClient->IOHandler = m_pIdIOHandlerStack;
}
else
{
m_pClient->IOHandler = NULL;
}
<other code>
m_pClient->Connect();
I got an exeption "403 forbidden" on "Connect"
proxy: 5.196.0.118::3128
I can connect to this server without proxy or ping it.
I used this proxy server successfully with my browser, but I can't use it for my code.
How can I resolve this problem?
This question could probably be related to doing anything high volume, but in this case I am trying to send emails.
I have setup the sending process in a new thread so user doesn't wait, and have overridden the request timeout to an hour.
The problem is that once the process get's up to about 2000 emails sent (looped over the below code about 2000 times) the server runs out of memory, stops responding, and needs a reboot.
Reading other topics on this, CF should be able to handle this volume of emails fine.
One thing I have considered is changing all object calls to straight DB Queries and using the cfmail tag to (I guess) remove all objects from being created and building up on reach request (which I guess is what is happening), but I'm not sure if that would make a difference, and really want to avoid that approach if possible. Something else I considered was splitting it across 3 or 4 seperate threads, but again, not sure if that would solve the problem.
Has anyone faced this problem, and what did you find worked to allow processing to continue without the ram slowly filling up and killing the server?
thread name="sendBroadcastEmail" rc="#rc#" prc="#prc#" filters="#rc.filters#" email="#email#" emailSignature="#emailSignature#"{
createObject( "java", "coldfusion.tagext.lang.SettingTag" ).setRequestTimeout(javaCast( "double", 3600 ));
//get profiles that it will be sent to
var sendToProfiles = profileService.getWithFilters(rc.filters);
var mailService = getPlugin("MailService");
var emailSent = false;
var sentCount = 0;
var failedCount = 0;
//send the email (and log in profile events)
if (listFind(attributes.rc.email.action,'send')){
for ( i=1; i<=arrayLen(sendToProfiles);i++){
var profile = sendToProfiles[i];
try{
if (len(trim(profile.getPrimaryEmail()))){
var emailBody = profile.processDynamicPlaceholders(attributes.rc.email.body);
var emailBody = attributes.emailSignature.getHeader() & emailBody & attributes.emailSignature.getFooter();
var sendEmail = mailService.newMail(
from = attributes.emailSignature.getEmailAddress(),
//to = profile.getPrimaryEmail(),
to = Application.settings.testemail,
subject = attributes.rc.email.subject,
body = emailBody,
type="html");
sendEmail.addMailParam(disposition='attachment', file=attributes.email.getAttachmentWithPath());
mailService.send(sendEmail);
//log profile event
profile.saveEvent(eventType = 3,
title="Broadcast Email: #attributes.rc.email.subject#",
description="Broadcast Email Sent: Subject: <br> #attributes.rc.email.subject#",
sentContent=emailBody,
ref2=1);
sentCount++;
}
}
catch (any exception){
//log profile event
profile.saveEvent(eventType = 3,
title="FAILED Broadcast Email",
description="<br>Subject: #attributes.email.subject#<br>This email should have been sent to this profile, but the attempted send failed. The likely cause is a malformed email address.",
sentContent=emailBody,
ref2=0);
failedCount++;
}
}
emailSent = true;
}
//persist email object
if (listFind(attributes.rc.email.action,'save')){
email.setTstamp(attributes.prc.now);
email.setSent(emailSent);
email.setStatsSent(sentCount);
email.save();
}
}//end thread
One approach would be to generate the emails in timed batches to spread the load evenly. The batch size and delay between batches can be adjusted to suit your environment.
thread name="sendBroadcastEmail" rc="#rc#" prc="#prc#" filters="#rc.filters#" email="#email#" emailSignature="#emailSignature#"{
createObject( "java", "coldfusion.tagext.lang.SettingTag" ).setRequestTimeout(javaCast( "double", 3600 ));
// set thread to a lowish prority
var currentThread = CreateObject( "java","java.lang.Thread" ).currentThread();
var priority = currentThread.getPriority();
currentThread.setPriority( 3 );
//get profiles that it will be sent to
var sendToProfiles = profileService.getWithFilters(rc.filters);
var mailService = getPlugin("MailService");
var emailSent = false;
var sentCount = 0;
var failedCount = 0;
//send the email (and log in profile events)
if (listFind(attributes.rc.email.action,'send')){
var emailsPerBatch = 1000; // divide into batches, set size here
var batchcount = Ceiling( ArrayLen( sendToProfiles ) / emailsPerBatch ); // number of batches
var batchdelay = 120000; // set delay between batches (ms)
// initialise first batch
var firstitem = 1;
var lastitem = emailsPerBatch;
for( var batch=1; batch<=batchcount; batch++ ) {
if( batch > 1 ){
// delay sending next batch and give way to other threads
currentThread.yield();
currentThread.sleep( batchdelay );
}
for ( var i=firstitem; i<=lastitem;i++ ){
var profile = sendToProfiles[i];
// generate emails ...
}
// initialise next batch
firstitem = lastitem++;
lastitem += emailsPerBatch;
if( lastitem > ArrayLen( sendToProfiles ) ) {
// last batch
lastitem = ArrayLen( sendToProfiles );
}
}
emailSent = true;
}
currentThread.setPriority( priority ); // reset thread priority
}//end thread
I am using apache commons httpclient version 4.2 to execute a simple web service based on SOAP 1.2. I am able to fire the web service properly through SOAP UI but I am unable to do the same using Java.
Following is the method I'm using to invoke the service.
private static byte[] callSOAPServer(String body, String SOAP_ACTION,
String SERVER_URL) {
byte[] result = null;
HttpParams httpParameters = new BasicHttpParams();
// Set the timeout in milliseconds until a connection is established.
int timeoutConnection = 15000;
HttpConnectionParams.setConnectionTimeout(httpParameters,
timeoutConnection);
// Set the default socket timeout (SO_TIMEOUT)
// in milliseconds which is the timeout for waiting for data.
int timeoutSocket = 35000;
HttpConnectionParams.setSoTimeout(httpParameters, timeoutSocket);
DefaultHttpClient httpclient = new DefaultHttpClient(httpParameters);
/*
* httpclient.getCredentialsProvider().setCredentials( new
* AuthScope("os.icloud.com", 80, null, "Digest"), new
* UsernamePasswordCredentials(username, password));
*/
HttpPost httppost = new HttpPost(SERVER_URL);
httppost.setHeader("soapaction", SOAP_ACTION);
httppost.setHeader("Content-Type", "text/xml; charset=utf-8");
System.out.println("executing request" + httppost.getRequestLine());
// now create a soap request message as follows:
final StringBuffer soap = new StringBuffer();
soap.append("\n");
soap.append("");
// this is a sample data..you have create your own required data BEGIN
soap.append(" \n");
soap.append(" \n");
soap.append("" + body);
soap.append(" \n");
soap.append(" \n");
/* soap.append(body); */
// END of MEssage Body
soap.append("");
System.out.println("SOAP Request : " + soap.toString());
// END of full SOAP request message
try {
HttpEntity entity = new StringEntity(soap.toString(), HTTP.UTF_8);
httppost.setEntity(entity);
HttpResponse response = httpclient.execute(httppost);// calling
// server
HttpEntity r_entity = response.getEntity(); // get response
System.out.println("Reponse Header:Begin..."); // response headers
System.out.println("Reponse Header StatusLine:"
+ response.getStatusLine());
Header[] headers = response.getAllHeaders();
for (Header h : headers) {
System.out.println("Reponse Header " + h.getName() + ": "
+ h.getValue());
}
System.out.println("Reponse Header END...");
if (r_entity != null) {
result = new byte[(int) r_entity.getContentLength()];
if (r_entity.isStreaming()) {
DataInputStream is = new DataInputStream(
r_entity.getContent());
is.readFully(result);
}
}
} catch (Exception E) {
System.out.println("Exception While Connecting " + E.getMessage());
E.printStackTrace();
}
httpclient.getConnectionManager().shutdown(); // shut down the
// connection
return result;
}
Following is my SOAP Request as well as the end point.
String soapRequest = "<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:typ=\"http://skash.service.sahaj.com/types/\">"
+ "<soap:Header/>"
+ "<soap:Body>"
+ "<typ:remoteSKASHDeductionElement>"
+ "<typ:vleId>?</typ:vleId>"
+ "<typ:paidAmt>?</typ:paidAmt>"
+ "<typ:refTxnId>?</typ:refTxnId>"
+ "</typ:remoteSKASHDeductionElement>"
+ "</soap:Body>"
+ "</soap:Envelope>";
String soapEndPoint = "http://xxx.xxx.xxx.xxx:xxxx/skashws/remoteSKASHDeductionSoap12HttpPort";
String soapAction = "http://xxx.xxx.xxx.xxx:xxxx//remoteSKASHDeduction";
// executeSOAPRequest(soapRequest, soapEndPoint, soapAction);
byte[] resp = callSOAPServer(soapRequest, soapAction, soapEndPoint);
System.out.println(IOUtils.toString(resp));
I can see that the namespace set to Envelope tag is for SOAP 1.2 and is well set. I am not sure where I'm going wrong. I am receiving the following version mismatch error.
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Header>
<env:Upgrade>
<env:SupportedEnvelope xmlns:soap12="http://www.w3.org/2003/05/soap-envelope" qname="soap12:Envelope"/>
</env:Upgrade>
</env:Header>
<env:Body>
<env:Fault xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<faultcode>env:VersionMismatch</faultcode>
<faultstring>Version Mismatch</faultstring>
<faultactor>http://schemas.xmlsoap.org/soap/actor/next</faultactor>
</env:Fault>
</env:Body>
you may use this code block
public class GenericSoapMessageFactory extends SaajSoapMessageFactory implements InitializingBean {
#Override
public SaajSoapMessage createWebServiceMessage(InputStream inputStream) throws IOException {
setMessageFactoryForRequestContext(soap11);
if (inputStream instanceof TransportInputStream) {
TransportInputStream transportInputStream = (TransportInputStream) inputStream;
if (soapProtocolChooser.useSoap12(transportInputStream)) {
setMessageFactoryForRequestContext(soap12);
}
}
SaajSoapMessageFactory mf = getMessageFactoryForRequestContext();
return mf.createWebServiceMessage(inputStream);
}
}
And web.xml params
<servlet>
<servlet-name>spring-ws2</servlet-name>
<servlet-class>org.springframework.ws.transport.http.MessageDispatcherServlet</servlet-class>
<init-param>
<param-name>messageFactoryBeanName</param-name>
<param-value>genericSoapMessageFactory</param-value>
</init-param>
</servlet>
I saw this error once and I changed the following line;
httppost.setHeader("Content-Type", "text/xml; charset=utf-8");
to
httppost.setHeader("Content-Type", "application/soap+xml;charset=UTF-8;");
and it fixed the issue. Try it.