Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
When looking up some information about web attacks, sql injection and cross-site scripting are always on the table. I can't imagine that such old web attack where there is a lot of information to find on the internet about how to prevent against it is still in the top 10 of most used web attacks? Any explaination for this?
I'll tell a story.
My mother used to volunteer with a group to go to the local college campus to help students register to vote (in the US, people can vote at age 18, but they aren't registered by default, they have to fill out a form). She and her group would set up a table in the quad with a supply of forms and guide the students to fill it out and mail it in.
After years of doing this, one of the other women in the group said, "We've been coming onto campus to help these kids register for TEN YEARS! When are they going to be able to do it on their own?"
My mom and the others looked at her and said slowly, "There is a new set of students turning 18 years old every year."
The same thing is true for defense against SQL injection and Cross-Site Scripting. There are new programmers entering the profession every year.
In fact, studies show that the number of software developers doubles every five years, which means at any given time, 50% of software developers are what I would consider "junior developers" with less than five years of experience. By the time those people have become senior developers, there's again just as many younger developers who have entered the profession after them.
All of them need to be trained to understand SQL injection and Cross-Site Scripting defense before they should be allowed to put their code on a live server.
One at a time.
Every year.
SQL injection and Cross-Site scripting will continue to be a thing as long as there are software developers.
I also can reference the SQLi Hall-of-Shame, a web page that references news stories about data breaches perpetrated by exploiting SQL injection vulnerabilities. The seem to be multiple such stories every month, and these are just the break-ins that made the news. It's undoubtedly the tip of the iceberg.
The quantity and the quality are two different beasts. "A lot of information" doesn't mean helpful information. On the contrary, there are many contradicting, open-ended and ambiguous recommendations.
For example, up to this day OWASP lists "Escaping All User-Supplied Input" as a Primary(sic!) defense. Which, frankly, a nonsense
Another example is a decades-old superstition that sounds as "escaping prevents SQL injections".
Given a lot of such misleading or open-ended recommendations, such us "always validate the user input" (without a single hint on which particular validation is meant) a junior dev gets extremely confused, and let an injection in.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Oh, Collective Wisdom of the Crowd,
I've been handed the Task. My inclination is to go code but as I'm old and weary I'm aware of my total ignorance in Web coding, as well as my tendency to code instead of using off-the-shelf parts (a.k.a. NIH) and the similarity of the Task to the problems solves with Wikis and Content Management Systems. So, my question is: Solve the Task using a Wiki, A Content Management System or roll my own site?
The Task:
I've videoed a three-days sports event of my Ninjutsu club, and have created a set of three DVDs, containing many "chapters". Most chapters consist of an explanation and demonstration of a technique, followed by the instructor moving around and correcting people.
The big Honcho would like some of the senior students to review the DVDs prior to production.
One way would be to reproduce a few sets of the DVDs, mail said students, and have them e-mail me their comments. This, however, is low tech, not sexy, and I'm sure would not generate the desired involvement.
As an alternative I thought about creating a web site for this purpose, with the added value that the web site would later, upon release, serve as a companion to the DVD. First-draft requirements appear to be:
Allow each reviewer to pick up the part of the material he’s "the owner of" (i.e. responsible for).
Provide a web page for each DVD chapter, together with a navigation system.
Upon creation each page will contain and embedded video of that chapter.
Allow each owner to mark her sections as “OK”, “With Issues” or “Remove”.
Direct reviewers to pages with sections having problems, or not-yet-reviewed, or with high activity (i.e. interesting).
Allow reviewers to collectively document the techniques demonstrated in the video sequences, especially during the corrections when the instructor can’t be clearly heard as the speakers are turned off. Upond release this documentation will be "frozen" and provide additional insight into the technique, in addition to what was provided in the event.
Generate the basis for sub-titles.
In addition to above documentation, each such page will also contain discussions between the reviewers concerning the technique. These discussions will be visible on the page, below the video and the documentation, unlike Wikipedia where the discussions appear on different tabs.
When documenting the techniques, the instructors will be able to create and use a collection of terms – names for the techniques. These names will be collected into a central ontology, together with their translation, and will later be used to index the content.
Hebrew support of the content is mandatory
The site will have the ability to contain translated versions of the content where the
user can choose the language she will use. So after release a Spanish speaking student who have purchased the DVD and gone to the site would be able to look at a Spanish translation of the documentation.
I know, I know this is a tall order and I'm only an egg :(
Stick with the email for the review; ninjas <3 email. Enforce participation through intimidation. Focus on shortening the time-to-release, IMO.
Use the time to figure out if creating an online back-end is worth it for the release--even a good martial arts video doesn't sell a lot of copies; if you're not Hatsumi or Hayes, even less.
It looks like the biggest requirement is I18N and comments.
I'd go with a Wiki; its collaborative model of content creation is perfect for things like this, and many support translations--although keeping up with the translating can be problematic. Wiki gardening is time-consuming and non-trivial, adding a layer of translation...
Although it'd give a whole new meaning to ninja edits.
Potential revenue or the emotional investment will dictate the scope of the project, but here's a couple of ideas to consider:
Ticketing system to allocate the work to users, track progress, define state of completion. I recommend the open source Request Tracker. This would be the easier option to implement in terms of management of the project, but doesn't touch on the l18n or the web development.
OR
A Component Content Management System to act as database and publishing tool. I would suggest the open source Pressgang CCMS. This would take more effort to implement but offers the features of Request Tracker with the addition of publishing output functionality (especially in terms of the use of DocBook XML and Publican). It is also built to work with the open source translation tool Zanata.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
My girlfriend is signing up for college courses, and she missed out on registration for a particular class that's ideal for the upcoming semester. All sections of the class are full. However, other students may drop/withdraw at any point, opening up a seat that she'd want to register for immediately.
I'm not aware of any auto-notification tool on the college's website. So, rather than reloading this page manually all day long over the next several weeks, I'd like to set something up to auto-notify her if a seat opens up.
What are the best/easiest options to do this?
This is the web page that I would need to monitor:
http://www6.austincc.edu/schedule/index.php?op=browse&opclass=ViewSched&term=212S000&disciplineid=PCVCD&yr=2012&ct=CC
Here is a screenshot of the specific numbers that I'm interested in:
https://skitch.com/troywarr/gptbe/acc-course-schedule-course-schedule-by-discipline-spring-2012
For any of the five course sections, when the first number (shown in the orange box) drops below 12, I'd like to know as soon as possible.
I envision this being some sort of web scraper that operates on short intervals (like every 5 minutes), checking for changes to the text within the appropriate HTML elements, then interfacing with an SMS system (ideally) or email.
This is important enough to warrant setting up a tool like this, but I don't have more than about a day to devote to it, so I'm hoping that such a tool or service already exists, and would greatly appreciate any recommendations. If not, any suggestions on what tools/languages/technologies to use would be great. Thanks!
It's already been done for you.
http://www.austincc.edu/register/waitlists/
Register for the waitlist and she'll get an email when a spot opens up.
It' standard fare at pretty much any educational institute. ;)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I am writing this to get a more realistic view of what I should be able to achieve.
I have been learning web development for just about a month now. Below are my level of proficiency on a scale of 1 to 10 (10 being not having to use google or ask a question on stackoverflow to solve all encountered issues)
Skills
Django: 5
jquery/ajax: 4.5 to 5
Html: 5
(just simple html code, having the "bone structure" of the site laid out without any fancy formatting or design)
using CSS: 0 (I think it's called css)
These are, from my understanding of the skills required to develop a site.
Am I a fast learner?
I started a month back at 0 in all categories (with limited to medium programming experience in other languages), I don't know if this counts as fast.
GOAL:
As a learning experience, I am trying to develop a news website where users subscribe to different news categories (ex: 'US news', 'Europe', 'Business'...) and they would get, in their news feed (a lot like facebook's news feed that automatically gets updated) feeds that are related to the categories they are subscribed to.
I haven't tackled website design yet; even though it is just for learning purposes, I would ultimately really like to have a nice design set up for the site, and deployed it on a server just so I would go through all the steps needed to actually launch a site.
I would really like to hear some feedback on feasibility/ get some insights on some of your personal professional experience on:
1- Hey is it feasible for a newbie to learn off the internet everything he needs to pull this off!!??
2- I am having a hard time putting a "deadline" to achieve this. How long will it take you to finish this? how long do you think it takes an almost complete novice to do this :)
Any other remarks/comments are welcome,
Thanks for sharing!!
-Rami
I think you may be over-rating your django skills a little there! I'd say someone is probably at 6/10, or 7, when they're relatively comfortable with the AOP parts of django - i.e. metaclasses, decorators, and so forth. That said, apologies if you are at this level!
I dare say what you'll find yourself doing is making the site so that it works - this won't take long at all - and then doing a ~huge re-write using more sophisticated code. And so on.
Another issue you may have with a dynamic site like this is database optimizations and, though not really applicable for a feeds-based site, caching. I'm currently working on doing optimizations on a large website, where the initial programmer didn't care about efficiency so long as it worked. So it became incredibly inefficient, some pages using 1000+ queries (though not bad code, at all, really) - some model methods potentially doing 50k+ queries (ouch!). Most optimizations were fairly trivial (select_related, annotate, aggregate, update, etc) - some were outside of the ORM's scope and required raw SQL and/or efficient c-based algorithms. The latter have sped some pages up from ~700queries/7seconds to 3queries in 0.4seconds - impossible to do in just django.
All in all, how long it'll take to make the website depends on your audience. If its for <100 users, then it won't take you long at all (apart from the design, I could spend years playing with css and my site would still look ugly) - if its intended for a much larger user-base, then you could be spending a lot longer on it... in my opinion!
1- Hey is it feasible for a newbie to
learn everything he needs of the
internet and pull this off!!??
Completely! The information is all there. Django is an extremely well documented framework. You might have to use the brain for specific actions but most of it already exists in the framework or by using plugins. The rest you already know. As for the CSS part, get a theme from some online site and adapt it to your needs. Keep backups to revert to in case you destroy something.
2- I am having a hard time putting a
"deadline" to achieve this. How long
will it take you to finish this? how
long do you think it takes an almost
complete novice to do this :)
Can't help you there.. No one works the same way and that will be based on your experience, knowledge, background and so on. Start doing it and adapt as you go.
Good luck!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Is It possible to perform a system analysis and design for a Website ( particularly a social Network ) ?
What are the Expected contents will be , In the document ?
can u provide an example , please ?
{ I made a social network (www.sy-stu.com) as to be my graduation project and I want to add a full analysis study to the graduation document , I do have experience in UML and Usecases just the Idea of an analysis of a website is not clear and never perform one before }
thanx in advance
This sounds very ambitious, but I'm sure it's possible. Unfortunately, I've forgotten a bit of System Analysis, but do adhere to many of its guiding principles for my own projects. In fact, I would say that most data-driven Web sites are excellent candidates for Systems Analysis and should be used always during Web planning for any project you plan on putting into production.
Straight from the wiki:
The development of a feasibility
study, involving determining whether
a project is economically, socially,
technologically and organizationally
feasible.
Conducting fact-finding
measures, designed to ascertain the
requirements of the system's
end-users. These typically span
interviews, questionnaires, or
visual observations of work on the
existing system.
Gauging how the
end-users would operate the system
(in terms of general experience in
using computer hardware or
software), what the system would be
used for etc.
For the first point, I would analyze different technologies such as ASP.NET, Ruby on Rails and PHP. Each technology has its strengths and weaknesses. One key thing to keep in mind is if you plan on making your social network free, you may consider open source technologies over proprietary - as many servers and application frameworks for proprietary projects are costly. I would also consider Web startup and hosting fees. If you plan on getting a reseller account with Host Gator, then you would need to factor in monthly billing costs. If you plan to host your own servers, you may be amazed at the cost of doing so. For a truly stable system, you would need to put a lot of work and cash into managing your own Web servers.
For the second point, you could probably locate plenty of information on user requirements from similar sites - just check out forums for DIY social networks and see what people are having issues with in the Technical Support section. Obviously, looking into technology based articles and magazines would be a good place to search on end user expectations - or even just joining Facebook and Twitter - see what they are doing since people seem content.
For the third point, again you can consult your competition and see how the user interface works out. Is it easy to use? Is it difficult in some aspects? If you had to use their system for 8 hours a day at least 5 days a week, what would drive you mad and how would you do it better? And keep in mind logical work flow as well. Knowing your user base is important too. In some systems, you may be developing for other programmers. Using strong jargon may be fine, but for a social network you must remember that they aren't familiar with Web site data flow and terminology. So your controls should still make sense to a computer novice and still work securely (don't forget system security too!) and in an organized fashion.
Finally, remember that things happen. I recently created a back-end site for a client of mine. I though the system worked very well - and they were very pleased, but I just got an email today that they want the way order items are stored to work differently. This is why there's a maintenance aspect to the System Development Life Cycle - things change after you finish deploying. It could also be said that if I had communicated with my client's needs more closely, this could have been resolved. Fortunately, the change is relatively minor, and we do live in a real world where things don't always work as we expect. We just do our best :)
As I said earlier, Systems Analysis is a lot of work and should be. The point of it is to determine that what you are trying to accomplish is feasible and practical without committing to a long term project that could span years. And always remember that no plan is perfect. If there were perfect plans, we wouldn't need new systems :).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I have an application which needs to get intraday stock quotes on several assets (indices, commodities etc').
I want to be able to query the data in HTTP and get it as CSV/XML format.
Now, I'd like to be able to ask the data provider for example what was the last bid/ask/price on GE (General Electric) at 4:00PM, and ask it in let's say 4:05PM on that day, for further processing.
Similar services to what I'm looking for:
Reuter's DataLink service can give me this data on the last trade of the day.
I need it to flow through all day long - intraday.
Yahoo Finance (the query formay within it) is a great service which does what I want in terms of data delivery yet I'm unsure regarding its reliability/timing since it's free.
Also, I couldn't find any information regarding the delay of the data they provide relatively from the real world timing (like many websites give this data in delay of ~20min).
QuoteRSS gives this for free as well, it let's me pick a ticket and get its data, yet once again I'm unsure regarding its reliability, as well as its timing, which I have doubt if this is "realtime" or close to that.
Finally this blog post by google "At long last, real-time stock quotes are here" claims to offer free data on certain stocks, but in Google Finance's pages I can't find anything about it, nor at their API pages, and again, who knows what delay I get from the realtime data.
In addition to the concern with the above mentioned services (Yahoo, QuoteRSS & Google) I'm not sure how/if they provide an intraday information regarding the stocks, something which I need.
Worth mentioning is that many websites which deal with Forex claim to be getting their data feed from Reuters/Bloomberg.
Didn't find such a solution on both's sites. I even went online with a sales rep. at Reuters to ask about it and his answer, after a decent discussion, was that "he's afraid he cannot offer me anything better than their service DataLink". How odd!!
So to summarize my question;
1) Where do I get such data feed, in which I select several tickets from several markets, and get a closer-than-20min information regarding these tickets, in concise format (CSV/XML)?
2) If Reuters/Bloomberg offer it (I'll probably also call them later) - where is it being offered, at their websites? I'd like to get the data from a "big name" such as these guys, for reliability reasons.
3) Regarding "realtime" or not, it depends on the cost. What costs should I prepare to? I'm assuming that realtime feed costs a LOT, so, is there an option between realtime and the 20min delayed feed? Something like 2-5min delay?
4) Please mention how, or if, I can query for stocks' data in a timely manner, like "what was the price of GOOG at 4:00PM?".
Note #1:
Please keep in mind, when answering, that I need the quotes intraday and not "by the end of the day".
Note #2:
If google/yahoo do actually offer this kind of service for free, how do I find it? Directly. I don't mind starting with these "freewares" for testing and such, especially if I can query for data in a timely manner as mentioned above ("what was the price of GOOG at 4:00PM?").
Note #3:
In terms of licensing, I do not intend to resell this information. Simply as that.
Before they closed shop, I used opentick. My blog post about opentick shutting down got quite a bit of traffic, so I decided to write another post that examined some potential opentick alternatives. Take a look at the companies in the post and comments. Hopefully one of them will work for you.
I have used IQFeed for some time. It is not HTTP or a CSV but it is a streaming push of ticks from their servers to you. The client is a bit kludgy but overall I find it to be acceptable for the price. This type of feed would be considered "realtime" by most people and since you are talking about minutes I assume that you are someone who is not worried about a couple seconds of latency here or there.
I have experience with Reuters (Thomson) feeds. They are expensive since we are now talking about TotalView/OpenBook data. This would be used to calculate the history of the order book and could be used for analyzing things like the liquidity of an equity at different price levels. I had a good experience with them at another job. 24/7 Engineering support, fixes, decent security db. The reality is that there is a wide variety of ways to get these feeds mostly from brokerages. I don't think this is what you are looking for since you mentioned things that were free.
There are "mid tier" providers like CQG although I have no experience with them.
In general no matter who you are using you need to be willing to implement their protocol and format. I have found this to be true no matter which feed I use. The good news is that all you need to do is make a parser.
What was the price of Google at 4:00PM? Who can say. Which part of 4PM? Would the price at 4PM would be something like the final print to the tape of the closing auction? Is it the auction midpoint? The price is what you can transact at which can be very different then what you see printed. ;-P
A final note: If you are building a trading system of some sort pay for your data. It should be cleaner than trying to assemble it. The exchanges charge for data and there is no real way around it. If you can't afford a couple of hundred bucks a month for some data then you probably don't have enough capital to be trading.
Concerning Bloomberg, I just called them & they said that they only provide market data for personal use. So you cannot show it on your site, but you can do whatever you want with it as long as you don't publish it.