I am on design number three I think now of a program that submits a series of stock tickers and metrics to Yahoo Finance. I don't need to go into too much total about what it does as I have got most of it up and running now apart from one remaining issue.
The Yahoo Finance site lists about 2700 stock tickers on the NASDAQ alone. I anticipated that submitting all of these in one filename URL statement might fall over for some reason, so set an initial string length of 500 tickers and built some nested macros to iterate through in 500 ticker blocks until everything I wanted had been extracted.
However during development of the code it seems that if I build a string with any more than about 200 tickers in I get an error telling me that SSL Support cannot be run and the code falls over.
Does anyone have any idea why this is? In ideal world I would like to be able to do this code in one pass where all 2700 stock tickers are pulled down. If this isn't possible if someone could explain why not that would be great.
Thanks
Related
At my work we have two systems, one that collects the customers payments automatically every month. And one that manages the memberships of those customers. Sadly our outdated technology doesn’t communicate to each other so we don’t know if a customer actually paid for their membership without manually auditing them.
I’ve been put in charge of this process and boy does it take awhile to do.
I have limited knowledge of C++ and was looking into maybe writing a program to do the comparisons for me.
I have two ideas on how to implement this, and was wondering what you guys thought. If these would be best or if it’s even possible or if there’s a better solution?
Current Setup: We have a list of all members in excel, with how much each should be paying, we then go through the actual money collected and check to make sure everyone’s payment went through and was processed and not declined.
Option 1: have a multi-dimensional array of strings. Read the excel file into this array it would have three Columns, first name, last name, amount they should be paying. This would be put in alphabetical order to help with the searching. I would then export the transactions in css file format and read each line one at a time. When it reads a line it would search the array for the same first and last name. Once found it would take the amount paid confirm it said processed and not declined and if so would subtract it from the customers amount they should be paying. In the end if every customers amount they should be paying is equal to 0 then everyone paid.
Option 2: is similar to option 1 just instead of using a multidimensional array it would use two css files. And not put the items into the array at the start.
Thoughts? Is this a smart way to combat this problem? I’m a newbie programmer so I’m just looking for suggestions/advice.
Your solutions would work, but are suited for small datasets. I don't now what your constraints are, but I think that a more elegant solution would be to setup a database on the first system first(instead of the excel file).
Are you allowed to create a database? How many customers are in the excel file?
I was trying to run the following code and I get variable number of tweets when I keep running the code at some interval of time (more than 15min). Sometimes I get 1400 tweets and 1200,1000,1600 tweets the other time. Can't I get fixed number of tweets all the time I run the code even if i change the keyword?
for tweet in tweepy.Cursor(api.search, q="#narendramodi", rpp=100).items(200):
You search does not specify any id limit.
Because of pagination, Twitter Search API looks for latest tweets every time you call it. Since tweets are added continuously, simple call to Search API returns the most recent ones and you'll get different number of tweets based on how many tweets were posted during the time you were querying. See Working with Timelines.
Please also note that Twitter Search API focuses on relevance rather than completeness of the results. See The Search API.
If you want to iterate over tweets, starting from the moment you run your application and continuing to older tweets, I recommend using max_id in your next query parameters setting it with the id field of the last result from your query as suggested here.
i want to list all available industries ( like: http://biz.yahoo.com/p/ ) and show all corresponding stocks.
Until now I'm using YAHOO.Finance.SymbolSuggest.ssCallback for the symbol suggestion and http://finance.yahoo.com/d/quotes.csv?s=... for getting the stock's data.
Does anyone have any idea how to get all industries and corresponding stocks?
Is there another hidden Yahoo API?
Lists of all available industries are called GICS Sectors for Standard and Poor's (S&P500 will use that) and ICB for Dow Jones and FTSE. Hence it used by Nasdaq, Nyse and others markets.
It seems like Yahoo uses a third industry classification by Morning Star, but since I'm not quite sure I will give both ways of retrieving data.
Morning Star
I don't know if Yahoo really sticks to this classification, but some names were really close so let's see it:
You need to go to their Index Data and in each sector, click on it and then at the bottom View complete index holdings.
It's not as precise as in Yahoo industry list, but it's all you can do with Morning Star. Not very convincing, I know...
GICS Sectors
GICS Sectors are now a trademark of Standard and Poor's and then data have to be sought for in S&P's website.
Short answer: take a look at this page, you will need to be registered (it's free and easy) and you can download spreadsheets (xls) with stocks and corresponding sectors. Nevertheless, things aren't always easy, and you will have to do a bit of a search to retrieve all stocks with their corresponding industries. For example, the file INDICATED_RATE_CHANGE.xls will give you some companies and their sectors in each month of 2012. Using that and SP500_DividendAristocrats_2012.xls you should be able to retrieve at least a large part of S&P 500 companies.
ICB
ICB is used by NYSE, NASDAQ etc... Then it's a lot simpler than S&P and MorningStar. Here is your answer. BOOM! Direct link!
Link is dead :(
Finally
I strongly advise you to use the simpler and most-used industry classification index: the ICB. It will always be available and publicly displayed since millions of investors relay everyday on it, without having to use S&P financial services or MorningStar brokerage services...
EDIT
You can look at nasdaq.com to retrieve all companies and their corresponding sector: here for Nasdaq and here for Nyse
Get all industry-IDs from here:
http://biz.yahoo.com/ic/ind_index.html
(look at the links)
Then use YQL ( https://developer.yahoo.com/yql/console/ )
with a query like this:
select * from yahoo.finance.industry where id=912
I understand SimpleDB doesn't have an auto increment but I am working on a script where I need to query the database by sending the id of the last record I've already pulled and pull all subsequent records. In a normal SQL fashion if there were 6200 records I already have 6100 of them when I run the script I query records with an ID greater than > 6100. Looking at the response object, I don't see anything I can use. It just seems like there should be a sequential index there. The other option I was thinking would be a real time stamp. Any ideas are much appreciated.
Using a timestamp was perfect for what I needed to do. I followed this article to help me on my way:http://aws.amazon.com/articles/1232 I would still welcome if anyone knows if there is a way to get an incremental index number.
I am trying to use the graph api with limit and since
I think the highest limit is 5000, so I am using that ( I want to make the fewest calls).
I am also trying to look 1 month back.
So I try:
https://graph.facebook.com/[ID of page]/feed&access_token=[accesstoken]&limit=5000&since=11-12-24
and I get 207 results, and the earliest date is december 24th, this is all fine, its saying hey there are only 207 results in the last month. The problem is there is a next link that has:
"next": "https://graph.facebook.com/[id of page]/feed?limit=5000&until=1324702511"
If I get this page, I start getting posts beore december 24th.
So my question is, how can I be sure I get all posts after a given date with fewest calls???
The kludge I am thinking of is to set the since on the first call to 1 day before, then if I get a post with that date, I know I got them all, if not I paginate... 5000 posts in one month is a lot, but I think its possible...
It seems like facebook should provide a way to get since with highest limit possible...I read this http://developers.facebook.com/blog/post/478/ but im still confused.