I'm working on an IoT application on top of embedded Linux, and I want to collect log data (mostly text files) of the devices.
The language I use is C++. I went through the documentation/tutorials of Firebase, however, it looks like only iOS, Android and Web (JS) are best supported, even the C++ part is assuming the carrier device is iOS/Android.
Is it's a good choice to use Firebase for my requirement? Should I just go ahead with C++ SDK or use REST API instead (that I can do with libcurl)
Thanks.
Depending on the complexity of what you want to do, you might just want to use the REST API.
Your biggest hurdle there is likely going to be the authentication part, once you get that out of the way, using the API itself is extremely simple.
Since you're talking about embedded Linux, your resources might be limited, which for me personally would be a reason to go use the REST API approach.
It comes down to ease of use (SDK), or lightweight (REST API). That's my 2 cts anyway...
Related
I like fast code execution (because of that I switched from Python to Go) and I do not like dependencies. Amazon recommends using SDK for simpler authentication (but in Lambda I can get tokens from IAM from environment variables) and because of built into SDK retry on errors (few lines of code, as I think). Yes it is faster to write my code using SDK, but what additional caveats about using pure HTTP API instead of SDK? Am I too crazy about milliseconds? Such optimizations worth it?
Anything you do with AWS is the result of an API call, whether executed by CLI, Web console, or SDK.
The SDKs make it easier to interact with those APIs. While you may be able to come up with some minor improvements for some calls, overall you will spend a lot of time doing it to very little benefit.
I think the stated focus on performance belies real trade-offs.
Consider that someone will have to maintain your code -- if you use an API, the test area is small, but AWS APIs might change or be deprecated; if you an SDK, next programmer will plug in new SDK version and hope that it works, but if it doesn't they'd be bogged down by sheer weight of the SDK.
Likewise, imagine someone needs to do a security review of this app, or to introduce something not yet covered by SDK (let's imagine propagating accounting group from caller role to underlying storage).
I don't think there is a clear answer.
Here are my suggestions:
keep it consistent -- either API or SDK (within given app)
consider the bigger picture (how many apps do you plan to write?)
don't be afraid to switch to the other approach later
I've had to decide on something similar in the past, with Docker (much nicer APIs and SDKs/libs). Here's how it played out:
For testing, we ended up using beta version of Docker Python bindings: prod version was not enough, and bindings (your SDK) were overall pretty good and clear.
For log scraping, I used HTTP calls (your API), "because performance", in reality comparative mental load using API vs SDK, and because bindings (SDK) did not support asyncio.
I'd like to know if there's something like a dll or lib (with headers, and possibly documentation) that can be linked into a normal* win32 C++ solution so it can (by function call) notify Application Insights of page views and send metric values.
This seems like it should exist and be simple to find, but I'm just going round in circles on Google, finding documentation for reading values back out via a web API, or using C#, or projects that other people tried in the past but now have no source available.
This is for updating a diagnostic tool which I'm trying to avoid having to completely re-architect because the logging now needs to go to a slightly different service.
*ie. not Winforms, C++/CX, Xamarin or UWP. Straight, command-line c++.
Currently there is no Application Insights client library for native applications (c/c++). Though it's in backlog I don't think it will be prioritized higher than java/node.js. So, I would not rely on its release anytime soon.
Customers who wanted to send telemetry from native applications usually picked one of the following options:
Using REST API directly. If application is going to use only couple of telemetry documents then it should be straightforward. With the most complexity coming from own requirements such as persistent channel.
Host CLR + use .NET Application Insights SDK
Have .NET service running on a box and implement communication channel between native app and this service.
Depending on environment (for instance, #2 makes sense if app already hosts .NET; #3 makes sense if there is already installed .NET service in addition to native app) you can check whether on of these options will work for you.
I have a requirement where I need to upload a large file (can be 10 GB) to a shared space(windows) ( say APP1) . and we have a separate application( say APP2) different network now I need to download the same file from in second application via internet.
My approach is I need to create webservice to upload the document to shared space. and then expose a webservice for outer world to download the document.
My point is how I can manage the huge files upload/download through webservice ?
Please suggest if some one have any idea. I have flexibikity to use any third party APIs.but the application can talk only through webservices.
From your question it's not really clear which development platform you mean, .NET, Java, etc.
Also it's important to know how interoperable your services should be, security requirements, etc. Anyway will try to come up with a couple of solutions which you might research in more detail if you found them useful.
.NET
It's relatively easy to built such a web service with WCF. It supports streaming which could be interoperable, reliable and secure to some extend. You can read more on this here. This approach implies you have a huge disk to store files and you have a good recovery plan for that in case it goes down or just dies.
.NET, Java, etc. - cloud based
There are a lot of vendors who provide cloud storage and APIs to work with it. It's an ideal solution for a quick start. They take care of data availability, redundancy, etc. All you have to do is to use their API to upload and download files and to pay them for this :) In many cases it's worth it. Personally I used to work with Amazon S3. Their API is simple to use and there's plenty of docs for that.
EDIT:
Amazon S3 provides a simple web-services interface that can be used to
store and retrieve any amount of data, at any time, from anywhere on
the web.
I think you should take a look at Amazon S3 overview here.
This also provides API for a number of different platforms - Java, .NET, Node.js, etc. You an find the full list here.
Hope it helps!
I am looking at developing a HTTP-based server for storing and later retrieving internal assets (just files/blobs of data). Clients should be able to connect with a simple HTTP request and retrieve data, or use a RESTful API to store files.
When a file is stored, we will need to make some calls into preexisting C++ code to update our asset database, set up internal linkages, and run various checks (e.g. is the data sane).
We have mountains of existing C++ tech here and it's not going to be practical to rewrite it in a high-level language, so we'd like to reuse as much of that as we can. However, in our brief research, it seems like putting together a performant HTTP server is absolutely trivial in higher-level languages, and we'd like to use as much off-the-shelf tech as possible for the HTTP server part.
Anyone have any experience/insight with this? We looked at Node.js, and it seemed very powerful, and then we looked at its C++ integration and got scared off. We aren't going to be able to rewrite all of our low-level tech to be non-blocking, and Node.js seems to require that. Others suggested Ruby, which we have some experience with, but we've never added C++ to the mix before. Totally open to other ideas as well.
Thanks guys!
I would use http://code.google.com/p/mongoose/ .
Mongoose is written in C and easily integrated into existing code.
Maybe you can try poco:
http://pocoproject.org/features.html
Poco is a good c++ library and they provide a HTTP server framework.
I'm looking to build my first web service and I would like to be pointed in the right direction right from the start.
Here are the interactions that must take place:
First, a smartphone or computer will send a chunk of data to my web service.
The web service will persist the information to the database.
Periodically, an algorithm will access and modify the database.
The algorithm will periodically bundle data and send it out to smartphones or computer (how?)
The big question is: What basic things do I need to learn to in order to implement something like this?
Now here are the little rambling questions that I've also got rolling around in my head. Feel free to answer these if you wish. (...maybe consider them extra credit?)
I've heard a lot of good things about RESTful services, I've read the wiki article, and I've even played around with the Twitter's webservice which is RESTful. Is this the obvious way to go? Or should I still consider something else?
What programming language do I use to persist things to the database? I'm thinking php will be the first choice for this.
What programming language do I use to interact with the database? I'm thinking anything is probably acceptable, right?
Do I have to worry about concurrent access to the database, or does MySQL handle that for me? (I'm fairly new to databases too.)
How on earth do I send information back? Obviously, if it's a reply to an HTTP request that's no problem, but there will be times when the reply may take quite a long time to compute. Should I just let the HTTP request remain outstanding until I have the answer?
There will be times when I need to send information to a smartphone regardless of whether or not information has been sent to me. How can I push information to my users?
Other information that may help you know where I'm coming from:
I am pretty familiar with Java, C#, C++, and Python. I have played around with PHP, Javascript, and Ruby.
I am relatively new to databasing, but I get the basic idea.
I've already set up my server and I'm using a basic LAMP stack. My understanding of L, A, M and P is fairly rudimentary.
Language: Python for it's ease of use assuming the GIL is not a particular concern for your requirements (e.g. multi-threading). It has drivers for most databases and supports numerous protocols. There are several web frameworks for it - the most popular probably being Django.
Protocols: if you are HTTP focused study SOAP and REST. Note, SOAP tends to be verbose, which causes problems moving volumes of data. On the other hand, if you are looking at other options study socket programming and perhaps some sort of binary format such as Google's protocol buffers. Flash is also a possibility (see: Flash Remoting). Note, binary options require users install something onto their machine (e.g. applet or standalone app).
Replies: if the process is long running, and the client should be notified when it's done, I would recommend developing an app for the client. Browser's can be programmed with JavaScript to periodically poll, or a Flash movie can be embedded to real time updates, but these are somewhat tricky bits of browser programming. If you're dealing with wireless phones, look at SMS. Otherwise I would just provide a way for clients to get status, but not send out notification (e.g. push vs. pull). As #jcomea_ictx wrote, AJAX is an option if it's a browser based solution - study jQuery.
Concurrency: understand what ACID means with regards to databases. Think about what should happen if you receive multiple writes to the same data - database may not necessarily solve this problem the way you'd want.
Please, for the love of programming, don't use PHP if you're already comfortable with Python. The latter makes for far cleaner, more maintainable code. Not that it's impossible to write good code in PHP, but it's a relative rarity. You can use Python for all the server-end stuff including MySQL interaction, with the MySQLdb module. Either with standard CGI, or FCGI, or mod_python.
As for the database, use of transactions will eliminate conflicts. But you can usually design a system in such a way that conflicts will not happen. For example, use of auto-incrementing primary-key IDs on each insert will make sure that every entry is unique.
You can "pull" data with Javascript, perhaps using AJAX methodology, or "push" using SMS or other technologies.
When replies take a while to compute, you can "poll" using AJAX. This is a very common technique. The server just returns "we are working on this" (or equivalent) with a built-in refresh until the results are ready.
I'm no expert on REST, but AJAX, especially when using polling rather than simply responding to user input, can be said to violate RESTful principles. But you can be a purist, or you can do whatever works. It's up to you.
I don't believe I've ever used any "push" technologies other than SMS, and that was years ago when many companies had free SMS gateways. So if you want to "push" data, better hope someone else joins in the conversation!
Use Java. The latest version of Java EE 6 makes coding RESTful and SOAP services a breeze, and it interoperates with databases very easily as well.
The benefits of using a true language instead of a script are: full server-side state, strong typing, multithreading, and countless other things that may or may not come in handy, but knowing they are available makes your project future proof.