I'm running a few webstores with Prestashop for the cart, but I have found a really cool service, but trying to figure out, how it can work for me though..
I've want to transform JSON to HTML on my store's backend before it reaches the customer within seconds since this is going to be a real-time update service for my descriptions and such.
I can send anyone that thinks they can help me the script that I have to use to get this data brought down.
Related
So the questions has more to do with what services should i be using to have the efficient performance.
Context and goal:
So what i trying to do exactly is use tag manager custom HTML so after each Universal Analytics tag (event or pageview) send to my own EC2 server a HTTP request with a similar payload to what is send to Google Analytics.
What i think, planned and researched so far:
At this moment i have two big options,
Use Kinesis AWS which seems like a great idea but the problem is that it only drops the information in one redshift table and i would like to have at least 4 o 5 so i can differentiate pageviews from events etc ... My solution to this would be to divide from the server side each request to a separated stream.
The other option is to use Spark + Kafka. (Here is a detail explanation)
I know at some point this means im making a parallel Google Analytics with everything that implies. I still need to decide what information (im refering to which parameters as for example the source and medium) i should send, how to format it correctly, and how to process it correctly.
Questions and debate points:
Which options is more efficient and easiest to set up?
Send this information directly from the server of the page/app or send it from the user side making it do requests as i explained before.
Does anyone did something like this in the past? Any personal recommendations?
You'd definitely benefit from Google Analytics custom task feature instead of custom HTML. More on this from Simo Ahava. Also, Google Big Query is quite a popular destination for streaming hit data since it allows many 'on the fly computations such as sessionalization and there are many ready-to-use cases for BQ.
I'm trying to help an animal shelter deliver faster updates when a new pet is added to their website. This is likely to happen between 0-20 times a day.
The website is a simple data dump, animals are in tables with row delineation (easy to parse) and have unique IDs. When a new pet is added, ideally this would trigger a mobile notification to subscribed users (could also be an email message). The faster updates are sent, the better, but checking every 30 mins or so would be fine. Because this is for a charity, I want to spend as little as possible on resources (because I also want to be able to scale this up for other shelters that might want to use this).
For instance mobile notifications, Twitter seems to be a good candidate. It looks like my needs wont run into fees/restrictions.
The part that I'm stuck on is how best to ping the site for updates and publish those updates to twitter. The two options I've come up with are:
Build my own system. Use a web crawler like Scrapy to periodically crawl the site and check for new petIDs. Using AWS, I think I could get by with a nano instance (~$57 a year). Using dynamoDB to cache existing petIDs seems like a small additional cost. Use twitter API to post updates
Use an RSS feed generator like Feedity. These seem to be pretty expensive: Feedity is $180/year for hourly updates and $390 for 15 minute updates. Has API integrated with Twitter.
I'd like to know if there are any better/simpler/cheaper/more obvious options I may be overlooking. Thanks!
I'm trying to develop a website that basically lets a user visit a page, and lets say click a button, and use their built in camera to live stream videos with audio to others that visit another url.
I need some clarity on what I need to develop, what I can get from 3rd party to save time. AWS looks to cover all the encoding and delivery http://aws.amazon.com/cloudfront/streaming/, but I'm confused on the process on which I should record and delivery the content to S3. Just to much information overload.
In all my research I looks like I should build a WebRTC, which I have done, then transport that data with javascript from the clients browser to my server, and thus to AWS. Is this the best format, or should I been using a 3rd party thats putting more time into that element?
I have seen the Kurento project, as well as this RecordRTC project.
Like I said, I'm finding there is just to much information overload on the topic.
So what are my options for:
In browser recording with WebRTC. Anything else I should do or just force users to roll up to a supporting browser?
WebRTC means I have to do Javascript for the delivery, is node a better option for the server to take delivery of this streaming data?
Anything else I need to know before I pass it off to S3 for delivery to the cloud front?
As you can see the core of my question comes within the recording and transporting the data to the web server so I can delivery it for streaming.
I am looking for the same thing.
In 2020, it seems it should be possible with RecordRTC and then uploading blobs / multiform data directly to S3.
I have a real time web analytics problem to address, and I'm wondering if some of the WSO2 products might be an appropriate solution.
An ecommerce web site shows pages of products to a browser user, and the web site vendor wants to collect details of what products were viewed in a list, what products were selected from the list for more info, what products were put into the basket, and what products were actually purchased - all in real time. I can use web page tagging to generate logging events for the four states (I.e. In list, view detail, in basket, purchased). The web site vendor wants too see results summarized by product and by rolling time band (e.g. Last hour, last 6 hours, last 24 hours, last 72 hours) by the four product states.
As a complete WSO2 newbie I'm hoping somebody can help with some pointers on how to address this. I've been reading about the BAM module to capture events. Is that a good place to start? Also can anybody suggest a good in memory data store to hold the event data aggregated by event type and rolling time period?
TIA
Yes, BAM is more kind batch processing, monitoring and complex engine and using it you can capture data, process and then present. In architectural point of view, the product states that are changed by the browser user will be captured by the web server and publish to BAM server.
A good point to start is learning about data publishing. Once you define the data [in BAM it is known as stream definition] to be published, you can write a hive script to process it and present. You can pump all data to BAM and then you can use hive script to process and store it in the manner you wanted. Later you can retrieve and present.
I've been trying to learn Django, but I'm still pretty much a web dev newbie, so please bear with me. Maybe something is just fundamentally wrong with this question...
For example, lets say some data exists in a JSON stream that is updated constantly. I'm trying to capture bits of that data and store it in my database, and it's displayed when I visit my Django built page. I guess there's two ways to do this:
In my views.py, it checks the data source, updates the database, and displays the information through a html file. This just seems like it's not the right way to do it. The source would be polled every time the page is viewed.
I would think the correct way to do it is have an application on the server that polls the data source every 1 minute or whatever and updates the database. The views.py only displays information from the database.
Am I even thinking about this correctly? I haven't found any information/examples on how to write the application that would sit on the server and constantly update the database.
Thanks!!
The second way is the right way to go about this, and the application that you would write to poll the json stream does not have to be written with django.
If you want to use the same models for the application, you can implement it as a custom management command, then run the command using cron at an interval. The command would poll the stream, update the database. Your view would then read the database and display the data.
If you want to do this in "realtime" (I use the word realtime here loosely), the server that is hosting the json stream should allow for "push" or a socket connection that will remain open.