I would like to do some real world data testing on micro cloud foundry but the capacity of postgres database is limited to 256MB which is not sufficient for my testing. Is there a way to increase the db capacity temporarily in offline mode for testing?
If not, can somebody point me the latest instructions of setting up private cloud foundry server on Ubuntu Server 12.04
you can ssh in to the instance, change the configuration for mysql and restart the service.
SSH to the MCF instance :
$ ssh vcap#api.<your-mcf-instance-name-here>.cloudfoundry.me
*note - if you can't remember the password for the vcap user, you can change this via the vm console menu by selecting option 3.
Edit the mysql-node configuration file :
$ vi /var/vcap/jobs/mysql_node/config/mysql_node.yml
the file should look something or exactly like this :
---
local_db: sqlite3:/var/vcap/store/mysql_node.db
base_dir: /var/vcap/store/mysql
mbus: nats://nats:f5dc63f74be5e38f#127.0.0.1:4222
index: 0
logging:
level: debug
file: /var/vcap/sys/log/mysql_node/mysql_node.log
pid: /var/vcap/sys/run/mysql_node/mysql_node.pid
available_storage: 2048
node_id: mysql_node_1
max_db_size: 256
max_long_query: 3
mysql:
host: localhost
port: 3306
socket: /var/vcap/sys/run/mysqld/mysqld.sock
user: root
pass: dc64fad710976ea5
migration_nfs: /var/vcap/services_migration
max_long_tx: 0
max_user_conns: 20
mysqldump_bin: /var/vcap/packages/mysql/bin/mysqldump
mysql_bin: /var/vcap/packages/mysql/bin/mysql
gzip_bin: /bin/gzip
ip_route: 127.0.0.1
z_interval: 30
max_nats_payload: 1048576
The two lines you are interested in are;
available_storage: 2048
and
max_db_size: 256
The first line is the maximum available amount of disk storage made available to MySQL, the second is the maximum size per mysql db instance. Set these to your desired values, obviously available_storage has to be large than max_db_size and also a multiple of that value.
Save the file and then restart the VM (shut it down via the menu in the VM console or do it via SSH) and you should be good to go!
Related
I've had to change hostname on a Google Cloud Compute that is running a WHM instance, but it keeps resetting every now and then and restart.
My /etc/hosts are currently as follow:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.156.0.7 cpanel.server-location-c.c.ascendant-hub-hidden.internal cpanel # Added by Google
169.254.169.254 metadata.google.internal # Added by Google
My System Information are:
Linux cpanel.xxx.com 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
My Old Hostname is something alike:
cpanel.xxx.com
I want my new hostname to become:
brain.xxx.com
Even when I change it from WHM using their Change Hostname feature, it keeps resetting.
Is their a cleaner method then setting a crontab?
Unfortunately, you're not able to change a custom hostname after you've created VM instance. Have a look at the documentation Creating a VM instance with a custom hostname:
You can create a VM with a custom hostname by specifying any fully
qualified DNS name.
and at the section Limitations:
You cannot change a custom hostname after you have created the VM.
To change this behavior you can try to file a feature request at Google Issue Tracker under this component.
UPDATE In addition, have a look at the documentation Storing and retrieving instance metadata section Default metadata keys:
Compute Engine defines a set of default metadata entries that provide
information about your instance or project. Default metadata is always
defined and set by the server. You can't manually edit any of these
metadata pairs.
and hostname is part of the default metadata entries and could not be changed manually.
UPDATE 2 As a possible workaround, you can use a startup script or other solutions to change the hostname every time the system restarts, otherwise it will automatically get re-synced with the metadata server on every reboot. For example, I applied this startup script via Custom metadata:
Key: startup-script
Value: #! /bin/bash
hostname changed-host-name'
then restarted VM instance and it works for me:
changed-host-name:~$ hostname
changed-host-name
These are few ways to change your hostname:
One way is to edit /etc/hostname directly - just switch file content with your new hostname.
The other way is to use hostnamectl set-hostname <your new hostname> which change /etc/hostname file for you.
But I think your problem is that Google keeps to overwrite some data not only when you reboot system but also while your VM is running. Assuming that above solutions, won't solve your issue.
Solution:
Thankfully Google Cloud Platform allows you to have custom hostname but you have to define them when creating new virtual instance. Check out this GCP document.
I have a web application running on Laravel5.2 framework, with session driver set to redis with following AWS setup.
Instance-1: Running web application, with Redis configurations in .env file as follow
Redis-host: aws-private-ip-of-instance-2
Redis-password: NULL
Redis-port: 6379
Instance-2: Redis-server running with following configuration
Bind aws-private-ip-of-instance-2 and 127.0.0.1
Working directory /var/lib/redis with 775 permission, and ower-group is redis.
RDB snapshot name dump.rdb with 660 permission, and ower-group is redis.
NOTE: In AWS inbound rule for port 6379 is configured for
Instance-2.
Everything works fine, until redis tries to write the data on the RDB file. Following error shows on front-end.
MISCONF Redis is configured to save RDB snapshots, but is currently
not able to persist on disk. Commands that may modify the data set are
disabled. Please check Redis logs for details about the error.
While in the logs of Redis server i got following data.
4873:M 23 Sep 10:08:15.028 * 1 changes in 900 seconds. Saving...
4873:M 23 Sep 10:08:15.028 * Background saving started by pid 7392
7392:C 23 Sep 10:08:15.028 # Failed opening .rdb for saving: Read-only file system
4873:M 23 Sep 10:08:15.128 # Background saving error
Things I have tried
Add vm.overcommit_memory = 1 to /etc/sysctl.conf, as suggested in Redis-administraition-blog
Change path to dump.rdb file to tmp folder and change permissions to 777.
This other Stack Exchange thread might help, since you are using a custom /tmp dir for data:
The simple way to do this is to run systemctl edit redis. This will create an override drop-in file /etc/systemd/system/redis.service.d/override.conf, in which you can place your changes (and the proper section):
[Service]
ReadWriteDirectories=-/my/custom/data/dir
You may also create that directory and place files ending in .conf in it manually. But do not leave the directory empty, as this will disable the service.
In either case, run systemctl daemon-reload and you are ready to restart your service.
Many threads also point to filesystem inconsistency as root cause. Since you are using EC2, check this AWS forums post:
To fix this, you will have to:
Stop the instance
Detach the root volume of your instance
Attach the volume as a data volume to any running Linux instance in the same availability zone
Perform a filesystem check (fsck) on the volume and fix the issues
Detach the volume and attach it back to your instance as it's root volume
Boot back instance and verify if the volume was able to mount successfully
As a last resort, terminate the instance if possible.
Hope it helps!
Well this is very embarrassing to post answer of own question, which was a really stupid mistake. But hope new folks here learns from my mistake too.
So first thing I have done is enable detail logs for redis-server in /etc/redis/redis.conf file by changing log_level option to debug.
Observe the logs and understand that my redis port 6379 was open for everyone on internet.
So from logs I observe that someone else's server is spoofing into my redis server and making it slave of it. And as my redis server is configure in a way that slave is read-only, when i try to access my redis-server it throw error of read-only.
After applying the fire-wall for redis server port, I have not encounter this issue anymore.
I'm new with the PCF and trying to deploy a simple web-app. I've installed the cf CLI, pcfdev and pushed my app to pcf:
cf push test-ui -b staticfile_buildpack
...
name: test-ui
requested state: started
instances: 1/1
usage: 256M x 1 instances
routes: test-ui.local.pcfdev.io
last uploaded: Thu 23 Aug 13:09:04 +03 2018
stack: cflinuxfs2
buildpack: staticfile_buildpack
start command: $HOME/boot.sh
state since cpu memory disk details
#0 running 2018-08-23T10:09:17Z 0.0% 5.3M of 256M 25M of 512M
So, now, I can access my test app by link test-ui.local.pcfdev.io from the same machine where I've started my pcf instance. But I don't know how to access to this app from another device in the same network.
Could someone tell me what I should do to open my test app from another device in the same network as my local machine?
Since PCFDev was installed in your local machine. I believe you cannot access the cloud foundry apps outside of your machine unless some networking stuff should be done that provide access to other machines.
I've found a solution: using the reverse proxy to redirect request from my local port app's url:
(using nginx as example):
server {
listen 8090;
server_name pcf-rp;
location / {
proxy_pass http://test-ui.local.pcfdev.io;
}
}
I'm a student from korea
first, i'm sorry about my low level english :)
I'm make a web service using AWS + nginx + django
I connect to AWS instance(ubuntu) using SSH protocol
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-74-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Sat Apr 30 07:03:51 UTC 2016
System load: 0.0 Processes: 105
Usage of /: 23.8% of 7.74GB Users logged in: 0
Memory usage: 14% IP address for eth0: 172.31.17.137
Swap usage: 0%
Graph this data and manage this system at:
https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
21 packages can be updated.
17 updates are security updates.
Last login: Sat Apr 30 07:03:52 2016 from 210.103.124.253
pyenv-virtualenv: no virtualenv has been activated.
and
manage.py runserver --settings=abc.settings.production
So everyone can access my web service!
but.... after 30miniute
the SSL connection is broken itself....
export this message
packet_write_wait: Connection to 52.69.xxx.xxx: Broken pipe
and nobody can't access my web service...
so... my web site can't access when my computer was power off, none SSL connection...
I want everyone can access my web service 24/7
please give me a method thank you :)
When you want to run a command that continues after your current shell terminates, you should use the nohup command to launch it.
That causes the process to be detached from its initial parent shell so it is not killed when the parent terminates.
I have to do distributed testing using JMeter. The objective is to have multiple remote servers in AWS controlled by one local server send a file download request to another server in AWS.
How can I set up the different servers in AWS?
How can I connect to them remotely?
Can someone provide some step by step instructions on how to do it?
I have tried several things but keep running into connectivity issues across networks.
We had a similar task and we ran into a bunch of issues as well. Here are the details of the whole process and what we did to resolve the issues we encountered. Hope it helps.
We needed to send requests from 5 servers located in various regions of the world. So we launched 5 micro instances in AWS, each in a different region. We chose the regions to be as geographically apart as possible.
Remote (server) JMeters config
Here is how we set up each instance.
Installed java:
$ sudo apt-get update
$ sudo apt-get install default-jre
Installed JMeter:
$ mkdir jmeter
$ cd jmeter;
$ wget ftp://apache.mirrors.pair.com//jmeter/binaries/apache-jmeter-2.9.tgz
$ gunzip apache-jmeter-2.9.tgz;tar xvf apache-jmeter-2.9.tar
Edited the jmeter.properties file in the /bin folder of the JMeter installation and uncomment the line containing the server.rmi.localport setting. We changed the port to 50000.
server.rmi.localport=50000
Started JMeter server. Make sure the address and the port the server reports listening to are correct.
$ cd ~/jmeter/apache-jmeter-2.9/bin
$ vi jmeter-server
Local (client) JMeter config
Then we set up JMeter to run tests remotely on these instances on our local client machine:
Ensured to use the same version of JMeter as was running on the servers. Installed Java and JMeter as described above.
Enabled remote testing by editing the jmeter.properties file that can be found in the bin folder of the JMeter installation. The parameter remote_hosts needed to be set with the public DNS of the remote servers we were connecting to.
remote_hosts=54.x.x.x,54.x.x.x,54.x.x.x,54.x.x.x,54.x.x.x
We were now able to tell our client JMeter instance to run tests on any or all of our specified remote servers.
Issues and resolutions
Here are the issues we encountered and how we resolved them:
The client failed with:
ERROR - jmeter.engine.ClientJMeterEngine: java.rmi.ConnectException: Connection - refused to host: 127.0.0.1
It was due to the server host returning the private IP address as its address because of Amazon NAT.
We fixed this by setting the parameter RMI_HOST_DEF that the /usr/local/jmeter/bin/jmeter-server script includes in starting the server:
RMI_HOST_DEF=-Djava.rmi.server.hostname=54.xx.xx.xx
Now, the AWS instance returned the server’s external IP, and we could start the test.
When the server node attempted to return the result and tried to connect to the client, the server tried to connect to the external IP address of my local machine. But it threw a connection refused error:
2013/05/16 12:23:37 ERROR - jmeter.samplers.RemoteListenerWrapper: testStarted(host) java.rmi.ConnectException: Connection refused to host: xxx.xxx.xxx.xx;
We resolved this issue by setting up reverse tunnels at the client side.
First, we edited the jmeter.properties file in the /bin folder of the JMeter installation and uncommented the line containing the client.rmi.localport setting. We changed the port to 60000:
client.rmi.localport=60000
Then we connected to each of the servers using SSH, and setup a reverse tunnel to port 60000 on the client.
$ ssh -i ~/.ssh/54-x-x-x.us-east.pem -R 60000:localhost:60000 ubuntu#54.x.x.x
We kept each of these sessions open, as the JMeter server needs to be able to deliver the test results to the client.
Then we set up the JVM_ARGS environment variable on the client, in the jmeter.sh file in the /bin folder:
export JVM_ARGS="-Djava.rmi.server.hostname=localhost"
By doing this, JMeter will tell the servers to connect to localhost:60000 for delivering their results. This ends up being tunneled back to the client.
The SSH connections to the servers kept dropping after staying idle for a little bit. To prevent that from happening, we added a parameter to each of the SSH tunnel set up directing the client to wait 60 seconds before sending a null packet to the server to keep the connection alive:
$ ssh -i ~/.ssh/54-x-x-x.us-east.pem -o ServerAliveInterval=60 -R 60000:localhost:60000 ubuntu#54.x.x.x
(.ssh/config version of all required SSH settings:
Host 54.x.x.x
HostName 54.x.x.x
Port 22
User ubuntu
ServerAliveInterval 60
RemoteForward 127.0.0.1:60000 127.0.0.1:60000
IdentityFile ~/.ssh/54-x-x-x.us-east.pem
IdentitiesOnly yes
Just use ssh 54.x.x.x after setting this up.
)
I just went though this on openstack and found the same issues... no idea why the jmeter remoting documentation only covers half the required steps. You can do it without tunnels or touching the properties files.
You need
All nodes to advertise their public IP - on AWS/OS this defaults to the private IP
Ingress rules for the RMI port which defaults to 1099 - I use this
Ingress rules for the RMI "local" port which defaults to dynamic. Below I use 4001 for the client and 4000 for servers. The port can be the same but note the properties are different.
If you are using your workstation as the client you probably still need tunnels. Above Archana Aggarwal has good tips for tunnels.
Remote servers
Set java.rmi.server.hostname and server.rmi.localport inline or in the properties file.
jmeter-server -Djava.rmi.server.hostname=publicip -Dserver.rmi.localport=4000
Sneaky server on client
You can also run one on the same machine as the client. For clarity I've set java.rmi.server.hostname but left server.rmi.localport as dynamic
jmeter-server -Djava.rmi.server.hostname=localip
Client
Set java.rmi.server.hostname and client.rmi.localport inline or in the properties file. Use -R etc like so:
jmeter -n -t Test.jmx -Rremotepublicip1,remotepublicip2 -Djava.rmi.server.hostname=clientpublicip -Dclient.rmi.localport=4001 -GmypropA=1 -GmypropB=2 -lresults.jtl
When you go for distributed testing using JMeter in AWS, I would suggest you to use docker - which will help us with jmeter test infrastructure very quickly. This way we can also ensure that same version of java and jmeter are installed in all the instances of amazon which is very important of JMeter distributed testing.
Ensure that - you set below properties and ports are open for jmeter-server. [they do not have to be 1099,50000 exactly]
server.rmi.localport=50000
server_port=1099
java.rmi.server.hostname=SERVER_IP
for client
client.rmi.localport=60000
java.rmi.server.hostname=SERVER_IP - this step is very important as the container in aws instance will have their own IP address in the docker network - so master and slave can not communicate. So we explicitly set this property
More info:
http://www.testautomationguru.com/jmeter-distributed-load-testing-using-docker-in-aws/