Replacing Google Site Search with AWS Cloudsearch - amazon-web-services

So I'm working on a site that has pretty specific global site search functionality that utilizes GSS which, as many of you already know, is going away in April. I need to crawl the site and send XML over to Cloudsearch, but I'm kind of confused as to how to go about this and I haven't found much material on building a global site search using AWS Cloudsearch after scouring the internet for a couple of days. So far, I'm planning on crawling the site with Apache Nutch, but I would really appreciate some input.

Did you come across our blog? Index the web with AWS CloudSearch Index the web with StormCrawler (revisited). I described how to use Nutch and StormCrawler to index to AWS Cloudsearch.
If you need the search to be hosted, I'd recommend Elasticsearch and Elastic Cloud instead. I found Cloudsearch slow, cumbersome and expensive and also there are more resources for Elasticsearch for StormCrawler and Apache Nutch.

Related

Setup Multiple Custom Domains for Clients using AWS

I want to give my customers an option to put their website (HTML, CSS, PHP (optional) ) in a custom domain.
By default, customers will get a basic subdomain like: their-site.mydomain.com
If they upgraded to Pro, I wanted to give an option to use custom domain by adding A record or CNAME.
After some google research, I found that lot of companies are using Amazon AWS for this task. However after checking different article, I did not get any idea how this would work with amazon or any other server.
All stuff needs to be done from the my Application UI by customers. There should be atleast 100s or 1000s of domains. I wanted to redirect each of them to their own website.
Any Idea how this will work? Please add all details if possible.

Extracting Sitecore Analytic Data

I'm new to Sitecore concepts, and I'm searching almost five days for an answer. I couldn't find what I'm looking for.
I'm trying to access Sitecore analytic data from a web service. I found a web service of Sitecore with using this document. I want to extract data that are about this. I believe this is a public demo site.
I want to access Web Api, then extract data and use it at my own project. Any idea?
When working with Sitecore 9, the API you want to use to connect to analytics data is known as xConnect. A secure connection is required using trusted certificates, so you cannot connect to an existing instance that somebody else setup like the Habitat demo you linked to.
xConnect is an abstraction API that allows you to collect and search all data in the xDB. The architecture fully supports both vertical and horizontal scaling of xConnect services separate from your Sitecore installations.
Resources
You can read more about xConnect here in the official developer documentation: https://doc.sitecore.net/developers/xp/xconnect/
There is an xConnect tutorial available here: https://doc.sitecore.net/developers/xp/getting-started/#tutorials-xconnect
I also have a small tutorial you can use on GitHub to start learning the concepts: https://github.com/jst-cyr/XConnectTutorial

Amazon product api import to DB

I've got access to the Amazon products API and I would like to create a website around it.
To create a website I thought on access the API and import the products information into database than use the information to create page and add a products description.
I have a few questions about it:
How I can save the relevant products into my database? I'm using PHP.
What is the best way to build this kind of website?
You got taken a wonderful initiative.
Below AWS technologies will help you to achieve in a scalable way.
DynamoDB:
To store data you can use DynamoDB,
https://aws.amazon.com/dynamodb/
To use with php,
http://docs.aws.amazon.com/aws-sdk-php/v2/guide/service-dynamodb.html
S3 with cloudfront:
To serve static webcontent you can use s3 and cloudfront.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/MigrateS3ToCloudFront.html
S3 PHP API:
http://docs.aws.amazon.com/aws-sdk-php/v2/guide/service-s3.html
You might not need API for cloudfront initially, manual configuration might work. But below is for reference,
http://docs.aws.amazon.com/aws-sdk-php/v2/guide/service-cloudfront.html
Be prepared to read a lot. Since you could build with some innovative product that could help millions of people.
Good Luck!

Is there a service that allows to access multiple API's using unified interface and one login?

Some time ago I was browsing the web, when I found a service that allowed to access multiple API's using single, unified interface and single login.
I remember that I browsed the catalog of API's and check OCR services to see what features they offer.
I don't remember if it was a free service or paid one. I didn't bookmark it and now I can't find it. I have found only API's catalog on Programmable Web.
Is anyone knows the name of this service?
Well, after getting one vote down I decided to google more. No results. I reviewed bookmarks and... bingo!
It's called mashape.com and what I have had in mind was this catalogue.
Disclaimer: I have no connection to this service. I just liked the idea.
Edit:
I have just found API search:
{API}Search.
It does not allow to access mutiple API's using single credentials, but might be usefull for API's discovery.

Amazon products API - Looking for basic overview and information

After using the ebay API recently, I was expecting it to be as simple to request info from Amazon, but it seems not...
There does not seem to be a good webpage which explains the basics. For starters, what is the service called? The old name has been dropped I think, and the acronym AWS used everywhere (but isn't that an umbrella term which includes their cloud computing and 20 other services too?).
There is a lack of clear information about the new 'signature' process. Gathering together snippets of detail from various pages I've stumbled upon, it seems that prior to August 2009 you just needed a developer account with Amazon to make requests and get XML back. Now you have to use some fancy encryption process to create an extra number in your querystring. Does this mean Amazon data is completely out of reach for the programmer who just wants a quick and simple solution?
There seems to be a tiny bit of information on RSS feeds, and you can get a feed of items that have been 'tagged' easily, but I can't tell if there is a way to search for titles using RSS too. Some websites seem to suggest this, but I think they are out of date now?
If anyone can give a short summary to the current state of play I'd be very grateful. All I want to do is go from a book title in my database, and use Classic ASP to get a set of products that match from Amazon, listing cover images and prices.
Amazon 'widgets' can display keyword search results on my pages, but I have less control over these, and they are shown to the user only - my code can't look inside them.
Your post contains several questions, so I'll try to answer them one at a time:
The API you're interested in is the Product Advertising API (PA). It allows you programmatic access to search and retrieve product information from Amazon's catalog. If you're having trouble finding information on the API, that's because the web service has undergone two name changes in recent history: it was also known as ECS and AAWS.
The signature process you're referring to is the same HMAC signature that all of the other AWS services use for authentication. All that's required to sign your requests to the Product Advertising API is a function to compute a SHA-1 hash and and AWS developer key. For more information, see the section of the developer documentation on signing requests.
As far as I know, there is no support for retrieving RSS feeds of products or tags through PA. If anyone has information suggesting otherwise, please correct me.
Either the REST or SOAP APIs should make your use case very straight forward. Amazon provides a fairly basic "getting started" guide available here. As well, you can view the complete API developer documentation here.
Although the documentation is a little hard to find (likely due to all the name changes), the PA API is very well documented and rather elegant. With a modicum of elbow grease and some previous experience in calling out to web services, you shouldn't have any trouble getting the information you need from the API.
I agree that Amazon appears to be intentionally obfuscating even how to find the API documentation, as well as use it. I'm just speculating though.
Renaming the services from "ECS" to "Product Advertising API" was probably also not the best move, it essentially invalidated all that Google mojo they had built up over time.
It took me quite a while to 'discover' this updated link for the Product Advertising API. I don't remember being able to easily discover it through the typical 'Developer' link on the Amazon webpage. This documentation appears to valid and what I've worked from recently.
The change to authentication procedures also seems to add further complexity, but I'm sure they have a reason for it.
I use SOAP via C# to communicate with Amazon Product API.
With the REST API you have to encrypt
the whole URL in a fairly specific
way. The params have to be sorted,
etc. There is just more to do. With
the SOAP API, you just encrypt the
operation+timestamp, and thats it.
Adam O'Neil's post here, How to get album, dvd, and blueray cover art from Amazon, walks through the SOAP with C# method. Its not the original sample I pulled down, and contrary to his comment, it was not an official Amazon sample I stumbled on, though the code looks identical. However, Adam does a good job at presenting all the necessary steps. I wish I could credit the original author.
I wrote a blog post on this subject, after spending hours wading through Amazon's obscure documentation. Maybe useful as another view on the process.
I found a good alternative for requesting amazon product information here: http://api-doc.axesso.de/
Its an free rest api which return alle relevant information related to the requested product.
Some links i found:
Forum thread for amazon tutorial request
Amazon Web Services
Some sort of script for using the amazon eCommerce API
another tutorial for amazon web-store-y stuff
Amazon and ebay e-commerce API tutorials
Straight from the horse's moutyh: Summary of Product Advertising API Operations which has the following categories:
Find Items
Find Out More About Specific Items
Shopping Cart
Customer Content
Seller Information
Other Operations
Since the time when the question was asked in 2009 the changes have, unsurprisingly, continued and some of the answers and links provided are now superseded or deadlinks.
As of February 2022, Amazon now provide the Product Advertising API Scratchpad for developers to try out API requests so they can get up and running in minutes:
Scratchpad is a tool to help Amazon Associates send basic requests to
the Product Advertising API. Follow the steps below and you can have a
working request with sample code in minutes.
The linked page also has onward links to pages where you may
sign up for the Associate program and Product Advertising API and access the complete API documentation.
As mentioned by #Reg Edit in his recent answer, Amazon now provides a scratchpad for their Product Advertising API, which in-fact does have a "SearchItems" endpoint which presumably returns products for a search query similar to the one a shopper would enter into Amazon's search bar while shopping.
Here's a link explaining on how to get access to Amazon's Product Advertising API. This would be helpful for anyone looking to display Amazon product's on their application programmatically.
In order to get access to Amazon's Product Advertising API, you must meet the following 3 requirements:
Have completed 3 sales in the last 180 days
Have an approved associates account
Comply with this agreement
Now if you don't meet the above requirements, the only other option Amazon gives you is to use their SiteStripe widget, which is a tool to help associates build links manually.
If you do not meet the requirements listed above and would still like to get Amazon product data for your app or website programmatically, you may use web scraping to achieve the same. Since the data is public, no one can legally stop you from scraping it. Depending on how experienced you are with programming, you could either build a scraper yourself or use a service that enables you to do so.
I have built one such service myself—it is called Amazon Product Search API and it allows users to grab search results from Amazon including product title, thumbnail, URL, etc. for any search query a user would make while shopping on Amazon. It supports all the major countries Amazon operates in.
Using this service does not require you to be an Amazon associate. Users may scrape up to 10k search results for free.