I have the following Target in my build script and it seems to work when I target the nuget feed for packages but my organization uses artifactory and has a private feed that requires credentials.
Target "RestorePackages" (fun _ ->
"./**/*.sln"
|> RestoreMSSolutionPackages (fun p ->
{ p with
Sources = "https://prd-artifactory.jfrog.com:8443/artifactory/api/nuget/some-private-feed" :: p.Sources
OutputPath = "./packages"
Retries = 4
ConfigFile = Some "./.nuget/nuget.config" })
)
I need to be able to pass in the username/password to this Target so I can run this on TeamCity passing in the credentials to use.
NuGet documentation states that you can run the following:
NuGet.exe Sources Add -Name <feedName> -Source <pathToPackageSource> -UserName xxx -Password <secret>
But I am not sure how to use this in my build script for the Target.
The Nuget source command you mentioned allows setting the credentials to access the given package source.
The credentials are added to the nuget.config file in the following manner:
<packageSourceCredentials>
<feedName>
<add key="Username" value="user" />
<add key="Password" value="...encrypted..." />
</feedName>
</packageSourceCredentials>
Just make sure you have the credentials in the nuget.config file you are referring to and it should work.
Related
I'm trying to publish artifacts to AWS CodeArtifact using sbt, but I'm having some troubles;
Given an sbt project, running the command sbt publish the package is uploaded to the repo but It remains in the Unfinished state. The AWS CodeArtifact documentation says:
Unfinished: The last attempt to publish did not complete. Currently only Maven package versions can have a status of Unfinished. This can occur when the client uploads one or more assets for a package version but does not publish a maven-metadata.xml file for the package that includes that version.
I'm using sbt at version 1.3.3, I'm not using plugins, the property publishMavenStyle is true.
I know that the sbt-maven-resolver (here the repo) solves the issue, but it seems an "unfollowed plugin", and moreover, using it I lose all logs about the publishing process, I don't trust it.
Did anyone have the same issue and have solved it somehow?
Using CodeArtifact with SBT
Setting up SBT with CodeArtifact
Publishing Packages with SBT (also avoiding the artifact being in Unfinished state.)
1. Setting up SBT with CodeArtifact
Create a CodeArtifact repository with a Maven upstream. For this example we're going to use repository maven-test in domain launchops
Open up the Connection Instructions in the console and choose mvn. We will need information from this later.
Copy the command which exports the "CODEARTIFACT_AUTH_TOKEN" environment variable from the console and run it in your environment. This will set the "CODEARTIFACT_AUTH_TOKEN" to be the password for our repository, the username is always aws.
In the build.sbt file import sbt.Credentials:
import sbt.{Credentials}
Now we need to setup the credentials. To do this we're first going to read the CODEARTIFACT_AUTH_TOKEN environment variable:
val repoPass = sys.env.get("CODEARTIFACT_AUTH_TOKEN").getOrElse("")
Next, we're going to use the previously imported sbt.Credentials to setup a new set of Credentials:
credentials += Credentials("launchops/maven-test", "launchops-123456789012.d.codeartifact.us-east-1.amazonaws.com", "aws", repoPass)
The values passed to the Credentials object are ("domain-name/repository-name", "repository hostname without protocol", "username", "password"), with username always being aws and password coming from the repoPass variable we only need to modify the first two to point to our repository.
Now we just need to instruct SBT to use our repository as a resolver. The consoles connection instructions will generate Maven settings, for example:
<repository>
<id>launchops--maven-test</id>
<url>https://launchops-123456789012.d.codeartifact.us-east-1.amazonaws.com/maven/maven-test/</url>
</repository>
We will use these values to create a resolver in our build.sbt file:
resolvers += "launchops--maven-test" at "https://launchops-123456789012.d.codeartifact.us-east-1.amazonaws.com/maven/maven-test"
The format of this is "resolvers += "ID From maven configuration in console" at "Repository URL from maven configuration in console".
To completely disable the use of public Maven repositories (Force CodeArtifact usage) you can add the following line to the build.sbt file:
externalResolvers := Resolver.combineDefaultResolvers(resolvers.value.toVector, mavenCentral = false)
After performing these setups steps you should be able to run sbt update and observe packages being downloaded through CodeArtifact.
Sample build.sbt for reference:
import sbt.{Credentials, Path}
name := "scala-test"
version := "0.3.0"
scalaVersion := "2.12.6"
organization := "com.abc.def"
val repoPass = sys.env.get("CODEARTIFACT_AUTH_TOKEN").getOrElse("")
credentials += Credentials("launchops/maven-test", "launchops-123456789012.d.codeartifact.us-east-1.amazonaws.com", "aws", repoPass)
resolvers += "launchops--maven-test" at "https://launchops-123456789012.d.codeartifact.us-east-1.amazonaws.com/maven/maven-test"
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.0.0" % "test",
"io.nats" % "jnats" % "2.0.0",
"org.json4s" %% "json4s-native" % "3.6.0"
)
2. Publishing Packages
Apart from pulling dependencies, SBT can also be used to publish packages. To have SBT publish to CodeArtifact we first need to set it up in the build.sbt file:
Add the following to the file:
publishMavenStyle := true
publishTo := Some("launchops--maven-test" at "https://launchops-123456789012.d.codeartifact.us-east-1.amazonaws.com/maven/maven-test")
At this point, technically, running sbt publish will push the package to CodeArtifact, however it will end up in Unfinished state. We need to make use of sbt-maven-resolver plugin to help get the package in the correct format: https://github.com/sbt/sbt-maven-resolver
In the project/plugins.sbt file add the following line:
addSbtPlugin("org.scala-sbt" % "sbt-maven-resolver" % "0.1.0")
Now you can run sbt publish and have the package publish to CodeArtifact successfully. If you see an error make sure that you are using a recent version of SBT.
You can achieve the same without using the sbt-maven-resolver plugin by following shariqmaws' answer without the plugin.
The publish will result in an artifact in "unpublished" state.
Then use the aws codeartifact cli to publish it (https://docs.aws.amazon.com/codeartifact/latest/ug/maven-curl.html)
I'm using react native for my project. On my old machine, when i ran amplify status, i had Auth, Api and Storage services listed.
I moved to my new machine, installed node, watchman, brew etc... and then navigated to my react native project and ran: react-native run-ios, and voila, my app is running. All the calls to my AWS Api, Auth and Storage are working perfectly.
Now i can make some amplify commands. Such as amplify status. I tried: amplify env add: here's what i got:
Users-MBP-2:projectname username$ amplify env add
Note: It is recommended to run this command from the root of your app directory
? Do you want to use an existing environment? Yes
? Choose the environment you would like to use: dev
Using default provider awscloudformation
✖ There was an error initializing your environment.
init failed
Error: ENOENT: no such file or directory, open '/Users/username/.aws/credentials'
at Object.openSync (fs.js:462:3)
at Proxy.readFileSync (fs.js:364:35)
at Object.readFileSync (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/util.js:95:26)
at IniLoader.parseFile (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/shared-ini/ini-loader.js:6:47)
at IniLoader.loadFrom (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/shared-ini/ini-loader.js:56:30)
at Config.region (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/node_loader.js:100:36)
at Config.set (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/config.js:507:39)
at Config.<anonymous> (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/config.js:342:12)
at Config.each (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/util.js:507:32)
at new Config (/usr/local/lib/node_modules/#aws-amplify/cli/node_modules/aws-sdk/lib/config.js:341:19) {
errno: -2,
syscall: 'open',
code: 'ENOENT',
path: '/Users/username/.aws/credentials'
}
Do you think credentials info needs to be brought/configured to my new machine?
When i run amplify configure project it's like doing an amplify init and building a project from scratch. I'm being asked:
? Enter a name for the project: ProjectName
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using (Use arrow keys)
angular
ember
ionic
react
❯ react-native
vue
none
etc....
I also already have a region, username and accessKey, secretAccess key etc..
I do not want to replace or ruin anything in my current backend or current project! Whats going on?
Ensure amplify-cli is installed and you're logged in with your AWS details.
npm install -g #aws-amplify/cli
amplify configure
Running amplify configure is mainly to give the cli knowledge of your AWS account so subsequent commands can have access to things.
If you get amplify: command not found errors try restarting your terminal. If still no luck, you will need to check amplify has been added to your PATH variable.
Run amplify env add , but choose an existing environment. This will let you choose the environment you created on your other machine so you can pull those settings down to your new machine.
amplify env add
? Do you want to use an existing environment? Yes
Production
Follow up with:
amplify pull
You don't need to run amplify add auth again or anything. All of that will pull down automatically after you've done the above.
You DO NOT need to do all config again, but some for sure
You have to install amplify cli npm install -g #aws-amplify/cli
use amplify pull
https://docs.amplify.aws/cli/start#amplify-pull
Follow the rest of steps -
-- provide the accessKeyId, secretAccessKey
-- region
-- select amplify project
and then rest of app related thing like IDE, directory......
I tried every solution then I found this. (in MacBook)
% sudo -i
Password:
~ root# npm install -g #aws-amplify/cli
-- Ctrl+D to exist from Root user
% amplify pull --appId xxxx --envName yyyy.
Note: To get --appId xxxx --envName yyyy
Log in to the AWS console. Choose AWS Amplify. Click your app. Go to Backend
environments. Find the backend environment you wish to pull. Click
Edit backend. See top right then click 'Local setup instructions
' ( amplify pull --appId
YOUR_APP_ID --envName YOUR_ENV_NAME )
Waiting until it request to verify your amplify.
✔ Successfully received Amplify Studio tokens.
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path: src
? Distribution Directory Path: build
? Build Command: npm run-script build
? Start Command: npm run-script start
✔ Synced UI components.
? Do you plan on modifying this backend? Yes
⠴ Building resource api/xxxx✅ GraphQL schema compiled successfully.
Edit your schema at ....
✔ Successfully pulled backend environment yyyy from the cloud.
✅
Successfully pulled backend environment staging from the cloud.
Run 'amplify pull' to sync future upstream changes.
% amplify pull
% npm install
% npm start
Hope this help every one!!
Happy Coding :)
I'm having trouble installing and configuring Leiningen on a Windows 10 work computer. I'm assuming that my company's firewall prevents the GitHub security certificate from authenticating.
The error I'm getting is:
Exception calling "DownloadFile" with "2" argument(s): "The request
was aborted: Could not create SSL/TLS secure channel." At line:1
char:145
+ ... che]::DefaultNetworkCredentials; $client.DownloadFile($a, $f)} "https ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : WebException
Failed to download
https://github.com/technomancy/leiningen/releases/download/2.8.1/leiningen-2.8.1-standalone.zip
It is possible that the download failed due to "powershell",
"curl" or "wget"'s inability to retrieve GitHub's security certificate.
The suggestions below do not check certificates, so use this only if
you understand the security implications of not doing so.
The PowerShell failed to download the latest Leiningen version.
Try to use "curl" or "wget" to download Leiningen by setting up
the HTTP_CLIENT environment variable with one of the following
values:
set HTTP_CLIENT=wget --no-check-certificate -O
set HTTP_CLIENT=curl -f -L -k -o
NOTE: Make sure to not add double quotes when setting the value
of HTTP_CLIENT
Github only supports TSL 1.2. By default PowerShell doesn't support this protocol.
To add TSL 1.2 support to every PowerShell session you need to edit your PowerShell profile: Microsoft.PowerShell_profile.ps1.
Path to profile (Windows 10):
C:\Users\%USERNAME%\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1
Add this line to profile:
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
After you add this line, try again with "lein self-install". At least this worked for me, remember to add PATH Variable.
Quick tutorial on how to add PATH variable:
Open up File Explorer
Right-click on This PC
Select Properties
On the left hand side of the new window select Advanced System Settings
On the bottom of this screen select Environment Variables…
Find the PATH variable
Append ;C\Lein\; (path to folder where is your lein.bat) to the existing PATH variable
If everything worked you should have similar answer:
I had the same problem.
If you have curl installed use the command it give you there:
set HTTP_CLIENT=curl -f -L -k -o
This worked for me.
When adding a custom jar step for an EMR cluster - how do you set the classpath to a dependent jar (required library)?
Let's say I have my jar file - myjar.jar but I need an external jar to run it - dependency.jar. Where do you configure this when creating the cluster? I am not using the command line, using the Advanced Options interface.
Thought I would post this after spending a number of hours poking around and reading outdated documentation.
The 2.x/3.x documentation that talks about setting the HADOOP_CLASSPATH does not work. They specify this does not work for 4.x and above anyway. Somewhere you need to specify a --libjars option. However, specifying that in the arguments list does not work either.
For example:
Step Name: MyCustomStep
Jar Location: s3://somebucket/myjar.jar
Arguments:
myclassname
option1
option2
--libjars dependentlib.jar
Copy your required jars to /usr/lib/hadoop-mapreduce/ in a bootstrap action. No other changes are necessary. Additional info below:
This command below works for me to copy a specific JDBC driver version:
sudo aws s3 cp s3://<your bucket>/mysql-connector-java-5.1.23-bin.jar /usr/lib/hadoop-mapreduce/
I have other dependencies so I have a bootstrap action for each jar I need copied, of course you could put all the copies in a single bash script. Below is .net code I use to get a bootstrap action to run the copy script. I am using .net SDK versions 3.3.* and launching the job with release label emr-5.2.0
public static BootstrapActionConfig CopyEmrJarDependency(string jarName)
{
return new BootstrapActionConfig()
{
Name = $"Copy jars for EMR dependency: {jarName}",
ScriptBootstrapAction = new ScriptBootstrapActionConfig()
{
Path = $"s3n://{Config.AwsS3CodeBucketName}/EMR/Scripts/copy-thirdPartyJar.sh",
Args = new List<string>()
{
$"s3://{Config.AwsS3CodeBucketName}/EMR/Java/lib/{jarName}",
"/usr/lib/hadoop-mapreduce/"
}
}
};
}
Note that the ScriptBootstrapActionConfig Path property uses the protocol "s3n://", but the protocol for the aws cp command should be "s3://"
My script copy-thirdPartyJar.sh contains the following:
#!/bin/bash
# $1 = location of jar
# $2 = attempted magic directory for java classpath
sudo aws s3 cp $1 $2
I am new to cloud foundry and am trying to find out of there is a way for downloading log files or search them in cloud foundry.
I know that I can open the logs files using vmc files but is there any other way of accessing the logs?
Thanks,
Kinjal
I think the easiest way to do this is using the VMC client library, 'cfoundry'.
The following ruby script connects and downloads the three main logs:
#!/usr/bin/env ruby
require 'rubygems'
require 'cfoundry'
creds = { :username => ARGV[0], :password => ARGV[1] }
app_name = ARGV[2]
files_to_dl = ['logs/staging.log', 'logs/stderr.log', 'logs/stdout.log']
c = CFoundry::Client.new "http://api.cloudfoundry.com"
c.login creds
app = c.app_by_name app_name
files_to_dl.each do |file|
begin
content = app.file(file)
local_path = file.match(/\/([^\/]+)$/)[1]
File.open(local_path, 'w') { |f| f.write(content) }
rescue CFoundry::NotFound
puts "404!"
end
end
This script assumes you are using the latest version of VMC (older, legacy versions don't use cfoundry) and that you also pass in username, password and application name when calling the script. It will write the contents of the remote files locally.