Connecting HDFS to Eclipse in HDP 2.2 - hdfs

I have installed HPD 2.2 on a Virtual box machine. Eclipse and hadoop eclipse plugin are already installed on Host system. I have mounted guest system root directory ( / ) using SSHFS and can successfully connect to it from the host nautilus.
After installing hadoop eclipse plugin i have changed the perspective to Map/Reduce mode. Also I have configured the hadoop install directory through the mounted guest os filesystem.
I need to configure eclipse to connect to hdfs (DFS location) I need Map/Reduce Master Host and DFS master host Port numbers. Please Reply with port numbers of Map/ Reduce master and dfs master so that my eclipse can browse the DFS filesystem.

Related

Do I need to install VMWare drivers to create a VMWare image using packe vmware-isor?

I am new to Packer and trying to build our environment from scratch automatically. I am getting below error when I try to create a VM using vmware-iso in Packer. So wanted to know if I need to install VMWare to run packer?
--> vmware-iso: Failed creating VMware driver: Unable to initialize any driver f
or this platform. The errors
from each driver are shown below. Please fix at least one driver
to continue:
* VMware application not found:
* VMware application not found:
* VMware application not found:
* VMware application not found:
*
You don't have VMware installed or on the path on your host machine. You need to install VMware Workstation etc.

How to remote debug python application running in Oracle virtual box?

For my Python + Django web project at workplace, we use Oracle Virtual box running Cent OS (Linux) clone image for all development and deployment. We use SSH clients namely mtputty and Mobaxterm depending on the developers choices.
We have checked out the project locally in windows and make it a shared folder for the clone running in Virtualbox. We are able to develop on Pycharm IDE by opening the files on shared folder.
This is the problem:
Though we are able to develop, we aren't able to remote debug on Pycharm IDE.
Is there a way to configure the remote debugging, so that when the server is running in the Virtual box, we can debug on the PyCharm.
Specs:
Python - 2.7
Django - 1.3.4
Pycharm - 4.5
I have done this by first:
1) Configuring the deployment and mapping in my project:
2) Then configure your Pycharm "Run/Debug Configurations", click on Run>Edit Configurations... click the plus sign and choose in the list "Django server".
In my configuration I'm forwarding my guest's port 8000 so I can access it on my host browser. You can do this on your vagrant file by adding:
config.vm.network "forwarded_port", guest: 8000, host: 8000

mkdir: cannot create directory : Protocol error : Virtualbox Shared folder

I am trying to create folders and file in Virtualbox shared folder from Host to guest.
But i get this error mkdir: cannot create directory : Protocol error.
Below are steps i performed to share folder
1:My host OS is Ubuntu and my guest is Ubuntu.
2:I attached a share folder to virtualBox VM
Folder Path:/DR/vault/config
Folder:config selected automount and make permanent.
3:In Guest OS i installed virtualbox guest additions
4:I am trying to mount folder on /mount/config path
I added entry in guest's /etc/fstab as
/config /mount/config vboxsf rw,uid=1000,gid=1000 0 0
Path gets successfully mounted after Guest reboot also i can see the files created in /DR/vault/config(Guest) to /mount/config(Guest) but i cant create folder or file in /mount/config (Guest)
Please suggest if anything is missing or if there is any other way.
In my case, I had the drive full, clearing some space on the drive solved the issue.
I have found this is a problem with filename lengths on DOS hosts.
I have been using VirtualBox VMs as a form of containerised environment for deploying to a JBoss server, having experienced far too many problems with Windows environments for said technology.
I was using a shared-directory with my host machine as I was trying to keep the virtual machine lightweight (i.e. keeping IDEs in the host machine), then checking code into the shared directory for deployment with Maven. However, maven was giving some odd errors when the directory length grew to over 255 characters. Try looking at your file absolute path (type "pwd") and seeing if it's longer than 255.
I've resolved this problem with the following:
On the guest machine, add your user to vboxsf group: sudo adduser username vboxsf
Restart the host machine.
My host OS is Windows, my guest OS Ubuntu.
This might be an indication that the folder does not have the correct permissions on the Virtualbox host.
E.g. my headless Virtualbox server is running as "vbox" user, so I had to give that user write permissions on the host folder.
I had a similar issue, turns out that for me it was the number of folders or length of path name involved in mkdir.
typically I had :
mkdir -p /projects/bot/node_modules/webpack/node_modules/uglifyjs-webpack-plugin/node_modules/uglify-js/node_modules/yargs/node_modules/cliui/node_modules/center-align/node_modules/align-text/node_modules/kind-of/node_modules/is-buffer
Manually I could create up to the last part of the path but not the 'is-buffer' folder
I am working in vagrant and my projects folder is a shared VM folder, maybe/probably windows' max path length is the reason.

Vagrant / Virtualbox, shared folder out of sync when there are many file-related operation

This is not about vagrant or virtualbox guest running slowly due to slow shared folder access, we know that can be resolved more or less by enabling nfs.
It's rather about mounted shared folder go out of sync when there are many file operations within the vm (enable nfs does not prevent it from happening) .
For example, when we are installing packages, like with php composer or node.js npm inside the vm, there is a certain probability that normal composer update or npm install will fail, and once it failed, only vagrant reload will help to restore the sync folder and allow the same command to pass without problem.
Such random failure only happens when executing on shared folder (nfs or not), so apt-get upgrade won't trigger the same problem as it runs within the vm folders.
Since the same sync problem does not appear when we run composer or npm from the host server, I am wondering what could have caused it and how do we go about debugging it?
Our vagrant setup and config:
if Vagrant::Util::Platform.windows?
config.vm.synced_folder "www", "/var/www", :extra => "dmode=777,fmode=777", :owner => "vagrant", :group => "vagrant"
else
config.vm.synced_folder "www", "/var/www", :extra => "dmode=777,fmode=777", :nfs => true
end
Guest: Ubuntu 12.04 LTS x64
Host: Windows 8, Mac OSX 10.8, Ubuntu 13 (yes, they all run into the same problem randomly)
Think we have more or less discover the source of problem:
Guest Addition version (4.1.x) that comes with our Ubuntu 12 LTS box does not match current Virtualbox version (4.2.x) installed on host machine. So file sync failed.
The easy fix:
run this command within vm sudo apt-get -y -q purge virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 to remove old guest addition
install vagrant vbguest plugin so future update is taken care of automatically during up: https://github.com/dotless-de/vagrant-vbguest

Change permissions of mounted subfolders

I'm using windows 7 as host and debian 6 in a VirtualBox VM. The guest additions are installed to share a folder provided by the host system.
In debian runs a webserver. So far everything is good.
But it's required to change the permissions on subfolders too. The shared folder is mounted with
mount -t vboxsf -o permissions,umask=0000,uid=33,gid=33 webdev /var/www
Is there any way to realize this?
The alternative is a samba share on the guest system, but that would be slow.