I'm trying to move about 100GB of data from one of my internal hosts up to our new AWS EFS volume.
My first inclination was to use rsync to get a trusted copy up to the volume, but I'm looking at somewhere around 8Mb/s and my first copy operation has taken somewhere around 24 full hours.
I read up on EFS File sync, a utility that's supposed to accelerate the copying operations of large datasets.
In the setup, the instructions dictate that I need to use a ESXi Virtual Image to launch a VM appliance that will connect up to AWS. I believe the recommendation is to use a hypervisor that can be assigned a reachable IP, but I only have my workstation to use.
I'm running into trouble configuring the appliance's network, so that it can handshake with the EFS Agent. I tried using a bridged adapter, but my corporate network uses AD and won't assign an IP to the VM.
Any suggestions?
Related
I'm new-ish to GCP and this is the first time I've run into a need to downsize a disk.
Background: I resized my boot disk from 1.5 TB to 12 TB to download a lot of data for a research project. But it turns out that the boot disk has an MBR partition type, which according to the GCP docs means "the maximum size of such disk would be 2 TB". Now I cannot use 10 TB out of 12 TB despite paying for it...
I created an additional disk of size 1.5 TB and used rsync -avxP /old/ /new/ to copy all files from the old boot disk to the new disk. Then I unmounted the old disk and tried to start the VM with the new disk as boot disk. But I'm getting an SSH error and cannot access my VM:
Code: 4003
Reason: failed to
connect to backend
Please ensure that:
your user account has iap.tunnelInstances.accessViaIAP permission
VM has a firewall rule that allows TCP ingress traffic from the IP range xx.xxx.xxx.x/xx, port: 22
you can make a proper https connection to the IAP for TCP hostname: https://tunnel.cloudproxy.app You may be able to connect without using
the Cloud Identity-Aware Proxy.
Has anyone encountered this before? How can I either use the 10 TB or solve the SSH problem? Thanks.
I have followed the suggestions at
Google Cloud How to reduce disk size?
How to mount new disk after reaching the 2TB limit Google Cloud
but have not been able to resolve my issue.
In one of our client scenario each laptop running a client software has to have a mapped S:\ drive.
Earlier they would map this S:\ drive to a drive in a Server on the same network like \\Server-name\$D
Now they are planning to move this server to EC2 in AWS. Is it possible to map the drive from EC2 server in local machine?
Local machine is in aclient network connected to AWS via VPN and DirectConnect.
In my research online (I am trying to map an Amazon EC2 drive to a local machine), I came to know that you can enable a VPN on EC2 server to make this happen. But I am looking for a inbuilt windows option as this is a as-is migration to AWS.
I need to host a service with rest-api on a server which does below listed tasks:
Download and upload files in s3 bucket
Run some cpu intensive computations
Return json response
I know an ec2 instance will be a better approach to host my service but given price differences between workspace and ec2 instance, I am exploring this route. Are there any limitations on amazon workspace that might prevent me from using them for my use case?
I came across ngrok which I believe can help me direct requests over the internet to my workspace local server.
Has anyone played around with it and could add some suggestion?
AWS terms of service do not allow you to do that I’m afraid. See section 36 on workspaces.
http://aws.amazon.com/service-terms/
36.3. You and End Users may only use the WorkSpaces Services for an End User’s personal or office productivity. WorkSpaces are not meant to accept inbound network connections, be used as server instances, or serve web traffic or your network traffic. You may not reconfigure the inbound network connections of your WorkSpaces. We may shut down WorkSpaces that are used in violation of this Section or other provisions of the Agreement.
I suggest you use an r5a.xlarge for the lowest cost 32GB RAM instance type (it’s AMD processor is cheaper than r5 on intel). Investigate whether spot instances would work if your state persists on S3 and not in the local instance, otherwise if you need it for at least a year reserved instances are discounted over on demand pricing.
I want to run a dpdk experiment using Amazon EC2 service. But there are a great number of services in AWS. I don't know which one to choose.
My experiment need two servers connected together using 10Gbps network adpater supporting dpdk. I run pktgen-dpdk on one server to send packets towards the other server. And another dpdk application will run in the other server to deal with these packets.
I think I can rent servers such c4.8xlarge c4.4xlarge. But I don't know how to set up the local network between them. The local network should have low latency.
Any suggestions will be appreciated! Thank you!
You're looking for Virtual Private Cloud (VPC). An AWS EC2 "instance" like your c4.8xlarge is just a machine. The VPC and several other components allow you to set up a broader network, routing, security groups (basically, a firewall) and other networking capabilities, including in your case a Gateway, which would allow your dpkg system to look out onto the Internet to find dependencies.
The in-network latency is extremely low, < 1ms in our experience. I think most current EC2 instances support 10Gbps networking and other speedy network capabilities.
Hi my goal is to create Active Directory in AWS. I used simple AD and used 2 public and 2 private subnets within the same VPC with the private ones being for the domain controllers. I created an EC2 instance within the same VPC with Windows Server so that I can manage the AD. My EC2 instance joins the domain with no problem. My problem however is I cannot get my local machines on my network to join the AD, as the DC's, are of course private IP's and I cant change the DNS on my machine to these IP's unless on the same network.
Im guessing I need a VPN to join my local network to the Network in the AWS cloud.
Is there a way to achieve having AD in AWS without a VPN such as using an elastic IP with NAT to communicate to the DC's? Or maybe even promoting my EC2 instance to a DC then connecting the local machines DNS to the EC2 instances elastic IP?
Any help is much appreciated and let me know if I am missing any information or not explaining the goal clear enough.
Your question mentions Simple AD. My comments will be for Active Directory in AWS.
Setting up Active Directory in AWS and on-premises is not as easy as I would like it to be. This topic can fill a small book or as Amazon does it, multiple hour long videos. Watch a few while thinking up your solution.
1) Simple AD is not real Active Directory. It is Samba 4, which is very good, but is an Active Directory clone.
2) Do not, and I repeat do not, think about putting Active Directory on a public IP address to serve your on-premises users. The number of ports that you need to open and the risk is just not worth it.
3) Most, if not all, real solutions for configuring Active Directory on-premises and in AWS involve VPNs. Either Direct Connect (DX), hardware routers (Cisco) or site to site VPNs built from OpenSwan or Windows Server.
Note: OpenSwan is very easy to setup, so this is the route I would recommend if cost is a factor. Otherwise look at Cisco ASA type routers (lots of vendors here) for your office and setup a VPN with IPSEC. If cost is not a factor, absolutely go with Direct Connect (DX).
Note: I also use OpenVPN to connect to AD in AWS from home. This setup routes my workstation to a VPC in AWS and is so easy to setup and use. You could start with this to get comfortable with networking to a VPC. There are preconfigured OpenVPN setups in AWS marketplace that are free (user limited).