What's the secret to getting ray workers to run within a certain directory, and within a certain virtual environment? I have the feeling I'm missing something fundamental:
I want to run ray tune for hyperparameter tuning (using aws). When I launch the head (and workers), I want to run ray from within a virtual machine (poetry, in my case). It just doesn't work at all. I tried:
setup_commands:
- cd /home/ubuntu/myimportantdirectory
- poetry shell
But then I get
2020-02-25 17:34:28,592 INFO updater.py:256 -- NodeUpdater: i-0893fd1914fd8743e: Running cd /home/ubuntu/myimportantdirectory on 18.207.133.151...
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
... which I guess means it fails, because then poetry fails with could not find a pyproject.toml file in /home/ubuntu or its parents
I also tried
setup_commands:
- cd /home/ubuntu/myimportantdirectory
- . path/to/poetry/activate
But then I get Command 'ray' not found, even though I'm certain it's there.
This works. But there's gotta be a better way. Also I can't believe this isn't documented. In the yaml, do this:
head_start_ray_commands:
- . /home/ubuntu/.cache/pypoetry/virtualenvs/robotics-7LCEAdjK-py3.6/bin/activate; ray stop
- ulimit -n 65536; . /home/ubuntu/.cache/pypoetry/virtualenvs/robotics-7LCEAdjK-py3.6/bin/activate; ray start --head --redis-port=6379 --object-manager-port=8076 --autoscaling-config=~/ray_bootstrap_config.yaml
worker_start_ray_commands:
- . /home/ubuntu/.cache/pypoetry/virtualenvs/robotics-7LCEAdjK-py3.6/bin/activate; ray stop
- ulimit -n 65536; . /home/ubuntu/.cache/pypoetry/virtualenvs/robotics-7LCEAdjK-py3.6/bin/activate; ray start --redis-address=$RAY_HEAD_IP:6379 --object-manager-port=8076
I'm trying to control a small car toy using the raspberry pi 3 the SSH
from my laptop.
The challenge I'm facing is, when I'm directly connected on RPi (mouse,
keyboard and monitor plugged to the RPI) everything works well, but
when I connect to RPi over the SSH connection, this is what I see as an
output: Xlib.error.DisplayNameError: Bad display name ""
The method I'm using to connect to my RPi is "ssh pi#"
Please can you advice me what to do next?
The sample code I'm using is shown below:
Thank you,
Johny 1984
from pynput import keyboard
from pynput.keyboard import Key, Listener
def on_press(key):
# DO SOMETHING
def on_release(key):
# DO SOMETHING
with keyboard.Listener(on_press=on_press, on_release=on_release) as listener:
listener.join()
the way I connecting to RPI over SSH
ssh pi#(RPI_IP_ADDRESS)
The error with the bunch of text above:
Xlib.error.DisplayNameError: Bad display name ""
It happens pynput needs X and some other setup listed in their documentation. So I ended up using keyboard package https://pypi.org/project/keyboard/ instead of pynput.keyboard.
https://unix.stackexchange.com/questions/427345/keyboard-monitoring-without-display
I modified his solution a bit so that I can exit from listening when escape key is pressed. Roughly something like this:
import keyboard
import time
#declaring it global so that it can be modified from function
global releaseListening
keepListening = True
def key_press(key):
print(key.name)
#if escape is pressed make listening false and exit
if key.name == "esc"
keepListening = False
keyboard.on_press(key_press)
while keepListening :
time.sleep(1)
I had a similar problem recently, and I could find a solution.
To make above work over ssh it is required to open /etc/ssh/sshd_config and uncomment the following lines if
they are commented:
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes
then on remote RPI type echo $DISPLAY. The result should be something like this:
localhost:10.0
when running/debugging your script, you should set env variable DISPLAY=result_of_echo_$DISPLAY
P.s Do not forget to connect via ssh using X server -> ssh user#address -X
I'm using openSuse as my desktop, prior to that i was using Ubuntu. My root (/) file system is btrfs and xfs for /home.
Whenever i try to run yast it is creating a pre and post snapshot even if there is no changes.
For example , If we are opening Hardware information in Yast. Which is not going to do any file system changes, except the harddisk attributes file modificaiton at /var/lib/smartmontools/
My Question is how do i tell to snapper ,Create snapshot only if there a real changes in certain folders (list of exclusion or list of inclusion). Because there are more snapshot. What is the optimal snapshot configuration.
I did the some changes in config /etc/snapper/configs/root Please correct me if anything wrong.
/etc/snapper/configs/root
# subvolume to snapshot
SUBVOLUME="/"
# filesystem type
FSTYPE="btrfs"
# btrfs qgroup for space aware cleanup algorithms
QGROUP="1/0"
# fraction of the filesystems space the snapshots may use
SPACE_LIMIT="0.5"
# users and groups allowed to work with config
ALLOW_USERS=""
ALLOW_GROUPS=""
# sync users and groups from ALLOW_USERS and ALLOW_GROUPS to .snapshots
# directory
SYNC_ACL="no"
# start comparing pre- and post-snapshot in background after creating
# post-snapshot
BACKGROUND_COMPARISON="yes"
# run daily number cleanup
NUMBER_CLEANUP="yes"
# limit for number cleanup
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="2-10"
NUMBER_LIMIT_IMPORTANT="4-10"
# create hourly snapshots
TIMELINE_CREATE="no"
# cleanup hourly snapshots after some time
TIMELINE_CLEANUP="yes"
# limits for timeline cleanup
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="0"
TIMELINE_LIMIT_DAILY="2"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="0"
TIMELINE_LIMIT_YEARLY="0"
# cleanup empty pre-post-pairs
EMPTY_PRE_POST_CLEANUP="yes"
# limits for empty pre-post-pair cleanup
EMPTY_PRE_POST_MIN_AGE="1800"
I am very new at scripting and I need some help. I have an unattended windows XP install that I created. We use two different Nvidia cards, about 50 of each, they use different drivers. I would like to be able to install the correct driver, based on the PNP Device ID. The script below outputs the PNP Device ID, next I want to capture the PNP Device ID and install the correct driver. I just need to be able to read the PNP Device ID, then run a silent install for the correct driver and software. I need to have the full Nvidia software installed, not just the driver. Any help would be greatly appreciated.
Paul
On Error Resume Next
strComputer = "."
Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")
Set colItems = objWMIService.ExecQuery _
("Select * from Win32_VideoController")
For Each objItem in colItems
For Each strCapability in objItem.AcceleratorCapabilities
Wscript.Echo "Accelerator Capability: " & strCapability
Next
Wscript.Echo "PNP Device ID: " & objItem.PNPDeviceID
Next
Const EVENT_SUCCESS = 0
I would recommend using Windows PowerShell for this, instead of VBscript. If you're new to scripting, then I'd suggest that learning PowerShell would be much more advantageous than learning VBscript. You'll be able to do a lot more, with a lot less code.
Let's move on to some code:
# If a video controller matches the string on the right of the -match operator ...
if ((Get-WmiObject -Class Win32_VideoController) -match 'VEN_10DE&DEV_0DFC') {
# ... then run this program
Start-Process -FilePath setup.exe -ArgumentList '/silent /log:"nvidia.log"';
}
I have multiple volumes (as nearly everybody nowadays): on Windows they end up specified as C:, D: and so on. How do I list these all like on a Unix machine with "ls /mnt/" with Powershell?
To get all of the file system drives, you can use the following command:
gdr -PSProvider 'FileSystem'
gdr is an alias for Get-PSDrive, which includes all of the "virtual drives" for the registry, etc.
Get-Volume
You will get:
DriveLetter, FileSystemLabel, FileSystem, DriveType, HealthStatus, SizeRemaining and Size.
On Windows Powershell:
Get-PSDrive
[System.IO.DriveInfo]::getdrives()
wmic diskdrive
wmic volume
Also the utility dskwipe: http://smithii.com/dskwipe
dskwipe.exe -l
Firstly, on Unix you use mount, not ls /mnt: many things are not mounted in /mnt.
Anyhow, there's the mountvol DOS command, which continues to work in Powershell, and there's the Powershell-specific Get-PSDrive.
Though this isn't 'powershell' specific... you can easily list the drives and partitions using diskpart, list volume
PS C:\Dev> diskpart
Microsoft DiskPart version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: Box
DISKPART> list volume
Volume ### Ltr Label Fs Type Size Status Info
---------- --- ----------- ----- ---------- ------- --------- --------
Volume 0 D DVD-ROM 0 B No Media
Volume 1 C = System NTFS Partition 100 MB Healthy System
Volume 2 G C = Box NTFS Partition 244 GB Healthy Boot
Volume 3 H D = Data NTFS Partition 687 GB Healthy
Volume 4 E System Rese NTFS Partition 100 MB Healthy
Run command:
Get-PsDrive -PsProvider FileSystem
For more info see:
PsDrive Documentation
PSProvider Documentation
This is pretty old, but I found following worth noting:
PS N:\> (measure-command {Get-WmiObject -Class Win32_LogicalDisk|select -property deviceid|%{$_.deviceid}|out-host}).totalmilliseconds
...
928.7403
PS N:\> (measure-command {gdr -psprovider 'filesystem'|%{$_.name}|out-host}).totalmilliseconds
...
169.474
Without filtering properties, on my test system, 4319.4196ms to 1777.7237ms. Unless I need a PS-Drive object returned, I'll stick with WMI.
EDIT:
I think we have a winner:
PS N:> (measure-command {[System.IO.DriveInfo]::getdrives()|%{$_.name}|out-host}).totalmilliseconds
110.9819
We have multiple volumes per drive (some are mounted on subdirectories on the drive). This code shows a list of the mount points and volume labels. Obviously you can also extract free space and so on:
gwmi win32_volume|where-object {$_.filesystem -match "ntfs"}|sort {$_.name} |foreach-object {
echo "$(echo $_.name) [$(echo $_.label)]"
}
You can also do it on the CLI with
net use
You can use the following to find the "total" disk size on a drive as well.
Get-CimInstance -ComputerName yourhostname win32_logicaldisk | foreach-object {write " $($.caption) $('{0:N2}' -f ($.Size/1gb)) GB total, $('{0:N2}' -f ($_.FreeSpace/1gb)) GB free "}
Microsoft have a way of doing this as part of their az vm repair scripts (see: Repair a Windows VM by using the Azure Virtual Machine repair commands).
It is available under MIT license at: https://github.com/Azure/repair-script-library/blob/51e60cf70bba38316394089cee8e24a9b1f22e5f/src/windows/common/helpers/Get-Disk-Partitions.ps1
If the device is present, but not (yet) mounted, this helps:
Get-PnpDevice -PresentOnly -InstanceId SCSI*