I have a dilemma. I do not know what is the best approach to the following scenario and then if it makes sense to invest time on developing a kernel module.
I have hardware (FPGA) that is exposed like many modules (around 30). Each module can be defined like:
Base address of the module;
Fields' offset (from base address);
The maximum number of fields per modules is around 10;
Each field has its own type like uint32_t, float32_t, uint32_t[] etc;
Some fields are read/write only and other read only;
Usually a module is ready as is. I mean that it is not necessary to implement any logic to check if it is possible to write to a field (except in few cases).
On the target device there is a custom Linux distribution (built from Yocto).
What do you think is better?
Application in user space that uses mmap (/dev/mem to map all
modules) and then reads/writes directly from/to memory. I have a C++
implementation and it is working but maybe it is not the best
solution... I need to set manually all offsets, using many
reinterpret_cast<> to read data properly and if something it is
wrong the application crashes;
Implement a character device
driver to expose each module like /dev/module1, /dev/module2 etc?
and use in user space open/write/read/release/ioctl. I have just
started to read a huge manual about Linux kernel development and I
am not so sure if a character device is a good idea here, especially
how to expose so many modules with so many fields to user space;
Other.
Thank you a lot for any ideas.
Using /dev/mem is quite straightforward, however it also causes some serious security issues. You either have to run your application as root or make the /dev/mem file accessible for other users, which are both unwelcome in designs that at some point will become products. If a malicious process can access the /dev/mem file it can possibly access any secret stored in RAM or corrupt any application - including the kernel itself. Even if your application is the only one able to access this file, any security concern of your code becomes the security concern of the whole system.
Preparing the driver is obviously not an easy task, but allows you to separate the (usually simple) privileged code from the applications in user space. In a simplest case you only have to provide some register read and write methods (through ioctl). These should check if the address is well aligned and constrained to the device address space. Additionally, the driver usually performs any additional address translation - so the client application does not need to know under which physical address was your device mapped (which is the case e.g. with PCI Express).
I would not recommend writing the driver from scratch, but to repurpose some existent code. In the mentioned case of PCI Express I have used two sources of inspiration - the Xilinx driver described here: https://www.xilinx.com/support/answers/65444.html (sources included) and more complicated 'pcieuni' and 'gpcieuni' from ChimeraTk project (https://github.com/ChimeraTK).
Related
I'm trying to do the following: Storing a text file (7kB) in the flash memory of a STEVAL-MKI109V2 (running with freeRTOS) board and read this text file and doing some computation with it on the device itself. I have 2 problems regarding that:
1) Storing the text file
Is it enough to just add the text file to my keil project? Can I access it after compiling?
2) Accessing the data
That's where I failed until now. At first I tried using fopen() from stdio.h but I got some errors on compilation. I found out that my project compiles by using microLib which seems it doesn't include file I/O. After compiling with standard C - library it was successful but as soon as I reach the fopen part in my code the system crashes.
Now I don't know if the reason is that the text file is not found or if I cannot use fopen() on my embedded system. I didn't find further information inside the STM documents or forums except the FLASH_Unlock(); function but it seems it's used for writing.
Do I need to store my text file in another way and access by memory address instead of just filename? I'm confused and cannot find any information online.
Thanks in advance for any help!
If you just want the contents of the file as a char-string, you can convert the file to C source code e.g. using a small Python program (or any other language, I just use Python for that as it is simple to do that in Python than in C or C++ for instance). Just create something like:
const char my_text[] = {
... here goes the text
};
Most simply, just embrace each line with ".
Then either add that file to the project (you'll require an extern declaration from where you use it) or #include it and make the declaration static (thanks #clifford).
If you simply want to embed a resource in your application the implementing a filesystem would be overkill and you should use #Olaf's method.
If however you want data that you can program independently of your application; then you could simply reserve the necessary number of on-chip flash pages and either program those separately via JTAG or add support for loading and programming the flash pages to your application. Or for greater flexibility, you could add a file-system that uses a reserved number of flash pages - that would also require you to add to your application a means to download and write the data.
All flash pages on the STM32F1xx are of equal size, so it does not matter whether you use low or high memory pages, but using the upper pages is simpler because the reset vector where your code starts is in the low memory. To reserve the pages (prevent the compiler placing code in them), you simply reduce the default upper address in the project's memory map options (I am assuming you are using Keil ARM-MDK/uVision since you mentioned MicroLib).
The both Keil's MicroLib or the its full featured library have support for I/O streams, but because the I/O capability of the target cannot be known in advance it requires what is known as retargetting. At it's simplest this is often implemented only for stdout/stdin streams, but you can implement file descriptors for any I/O device - however to perform file I/O you need a file-systems such as ELM FatFs or Yaffs for which you will still need to implement the low-level drivers for accessing the flash. If you use a file-system library; you do not actually have to hook in stdio via retargetting; you can access the library directly - I mentioned retargetting because you seem to have a somewhat loose grasp of how a stdio works.
The details of flash programming on the STM32F1xx are in a separate manual from the main reference, while the STM32 standard peripheral library includes low-level functions to support programming. Here you will find a serious gotcha not made clear elsewhere in ST's documentation; when you program or erase flash, it locks-out the address and data bus to the entire flash memory - since that is normally where the processor is also fetching instructions from, the entire core stalls for the duration of the operation, which can be as much as 40ms (it is worse on STM32F2xx at 800ms!); consequently writing to a flash page may mess up time critical operations.
If you want to use a filesystem on such a device; you may be better off using an SPI port to communicate with an SD card, or otherwise using off-chip non-volatile memory.
I know the question might seem a little vague but I will try to explain as clearly as I can.
In C++ there is a way to dynamically link code to your already running program. I am thinking about creating my own plugin system (For learning/research purposes) but I'd like to limit the plugins to specific system access for security purposes.
I would like to give the plugins limited access to for example disk writing such that it can only call functions from API I pass from my application (and write through my predefined interface) Is there a way to enforce this kind of behaviour from the application side?
If not: Are there other language's that support secure dynamically linked modules?
You should think of writing a plugin container (or a sand-box), then coordinate everything through the container, also make sure to drop privileges that you do not need inside the container process before running the plugin. Being run in a process means, you can run the container also as a unique user and not the one who started the process, after that you can limit the user and automatically the process will be limited. Having a dedicated user for a process is the most common and easiest way, it is also the only cross-platform way to limit a process, even on Windows you can use this method to limit a process.
Limiting access to shared resources that OS provides, like disk or RAM or CPU depends heavily on the OS, and you have not specified what OS. While it is doable on most OSes, Linux is the prime choice because it is written with multi-seat and server-use-cases in mind. For example in Linux you can use cgroups here to limit CPU, or RAM easily for each process, then you will only need to apply it for your plugin container process. There is blkio to control disk access, but you can still use the traditional quote mechanism in Linux to limit per-process or per-user share of disk space.
Supporting plugins is an involved process, and the best way to start is reading code that does some of that, Chromium sand-boxing is best place I can suggest, it is very cleanly written, and has nice documentation. Fortunately the code is not very big.
If you prefer less involvement with actual cgroups, there is an even easier mechanism for limiting resources, docker is fairly new but abstracts away low level OS constructs to easily contain applications, without the need to run them in Virtual Machines.
To block some calls, a first idea may be to hook the system calls which are forbidden and others API call which you don't want. You can also hook the dynamic linking calls to prevent your plugins to load another DLLs. Hook disk read/write API to block read/write.
Take a look at this, it may give you an idea to how can you forbid function calls.
You can also try to sandbox your plugins, try to look some open source sandbox and understand how they work. It should help you.
In this case you really have to sandbox the environment in that the DLL runs. Building such a sandbox is not easy at all, and it is something you probably do not want to do at all. System calls can be hidden in strings, or generated through meta programming at execution time, so hard to detect by just analysing the binary. Luckyly people have already build solutions. For example google's project native client with the goal to generally allow C++ code to be run safely in the browser. And when it is safe enough for a browser, it is probably safe enough for you and it might work outside of the browser.
I want to make a program for some people, so i make the program again for each person, and if someone gives the program to other guy, he can't use it.
How can I do that?
Without any internet connection.
Does any PC has some unique id or something like that, that i can make him a program to get it, so he will send me, and in my program i'll check if is the same, if not the program will stop.
Something like hwid will work?
Is the hardware id unique and cannot be changed?
If so, how can I get it? I found a lot of questions, but without any good answers..
Take a look at these:
Uniquely identify PC based on software/hardware
C++ API : license management to protect a software
Generating a Hardware-ID on Windows
Restrict functionality to a certain computer
How to get unique hardware/software signature from a windows pc in c/c++
If you want something a bit harder to spoof than whatever the machine itself can tell you, you'll probably need a USB dongle dedicated for this purpose.
There are several ways to identify a computer from where a program runs:
WMI - Windows provides a set of classes that can be used for most hardware enumeration and identification tasks, which is named WMI or Windows Management Instrumentation. These are extensions to the Windows Driver Model (WDM).
CPU ID - The solution that seems to be the best choice is to sample the CPU unique identification number (or CPU ID). However, there are several problems that makes it impossible to rely on reading the CPU ID.
To begin with, most CPUs with the exception of the old Pentium III, don't have a unique CPU Serial Number. Intel has removed this feature for privacy reasons.
It is still possible to generate a unique ID from the motherboard as a whole. That certainly works but the huge number of different types of motherboards and manufacturers makes it next to impossible to generate a unique ID that will cover all of them.
MAC address based hardware ID
The next choice for obtaining such a unique ID would be sampling the MAC address. To begin with, what is the "MAC address"? It stands for Media Access Control. The MAC address is 48 bits long (6 bytes). The GetMACAddress code sample explains how to obtain the MAC address.
However, there is one problem with this approach: the MAC address can be easily changed into a new one...
Hard Drive serial number
It seems that the only reliable solution for obtaining a machine ID would be using the serial number of the main Hard Drive. The second example, GetHDSerialNumber, shows how to obtain this ID. From my experience, this approach is the best one and the most reliable for generating a unique machine based hardware ID.
See also this article.
Like others have said, it is really hard to do this reliably.
You CAN use things like hardware dongles or licensing software to try to restrict use. For anyone sufficiently motivated this is a speed bump, not much more.
Another aspect of this is that the more secure you try to make it, the higher the risk that it'll be too restrictive. That is, it might end up accidentally blocking legitimate use, which is a really bad thing to do if you want to keep users happy.
This was tried many, many times when the PCs became popular. Each time a dismal failure. It even interferes with rights the law grants the user (keep backup copies). It also turned out that the hassle for the user was enough for many of them to just don't use the "copy protected" programs.
Today this is done successfully by the various gaming consoles, but there the provider of the console has a very tight control over the machine and the software. By force, those can't be used as regular computing platforms by the user, they are single-purpose. No wide range of software available.
The only ones to pull of this feat on regular machines have been expensive programs like Mathlab or Autocad, mostly through some sort of "license server" under tight control of the network administrator, tied to the specific server on which it runs by some long-winded procedure. And even so, it isn't too hard to get pirated ("unlocked") copies.
Disclaimer: I apologize for the verbosity of this question (I think it's an interesting problem, though!), yet I cannot figure out how to more concisely word it.
I have done hours of research as to the apparently myriad of ways in which to solve the problem of accessing multi-GB files in a 32-bit process on 64-bit Windows 7, ranging from /LARGEADDRESSAWARE to VirtualAllocEx AWE. I am somewhat comfortable in writing a multi-view memory-mapped system in Windows (CreateFileMapping, MapViewOfFile, etc.), yet can't quite escape the feeling that there is a more elegant solution to this problem. Also, I'm quite aware of Boost's interprocess and iostream templates, although they appear to be rather lightweight, requiring a similar amount of effort to writing a system utilizing only Windows API calls (not to mention the fact that I already have a memory-mapped architecture semi-implemented using Windows API calls).
I'm attempting to process large datasets. The program depends on pre-compiled 32-bit libraries, which is why, for the moment, the program itself is also running in a 32-bit process, even though the system is 64-bit, with a 64-bit OS. I know there are ways in which I could add wrapper libraries around this, yet, seeing as it's part of a larger codebase, it would indeed be a bit of an undertaking. I set the binary headers to allow for /LARGEADDRESSAWARE (at the expense of decreasing my kernel space?), such that I get up to around 2-3 GB of addressable memory per process, give or take (depending on heap fragmentation, etc.).
Here's the issue: the datasets are 4+GB, and have DSP algorithms run upon them that require essentially random access across the file. A pointer to the object generated from the file is handled in C#, yet the file itself is loaded into memory (with this partial memory-mapped system) in C++ (it's P/Invoked). Thus, I believe the solution is unfortunately not as simple as simply adjusting the windowing to access the portion of the file I need to access, as essentially I want to still have the entire file abstracted into a single pointer, from which I can call methods to access data almost anywhere in the file.
Apparently, most memory mapped architectures rely upon splitting the singular process into multiple processes.. so, for example, I'd access a 6 GB file with 3x processes, each holding a 2 GB window to the file. I would then need to add a significant amount of logic to pull and recombine data from across these different windows/processes. VirtualAllocEx apparently provides a method of increasing the virtual address space, but I'm still not entirely sure if this is the best way of going about it.
But, let's say I want this program to function just as "easily" as a singular 64-bit proccess on a 64-bit system. Assume that I don't care about thrashing, I just want to be able to manipulate a large file on the system, even if only, say, 500 MB were loaded into physical RAM at any one time. Is there any way to obtain this functionality without having to write a somewhat ridiculous, manual memory system by hand? Or, is there some better way than what I have found through thusfar combing SO and the internet?
This lends itself to a secondary question: is there a way of limiting how much physical RAM would be used by this process? For example, what if I wanted to limit the process to only having 500 MB loaded into physical RAM at any one time (whilst keeping the multi-GB file paged on disk)?
I'm sorry for the long question, but I feel as though it's a decent summary of what appear to be many questions (with only partial answers) that I've found on SO and the net at large. I'm hoping that this can be an area wherein a definitive answer (or at least some pros/cons) can be fleshed out, and we can all learn something valuable in the process!
You could write an accessor class which you give it a base address and a length. It returns data or throws exception (or however else you want to inform of error conditions) if error conditions arise (out of bounds, etc).
Then, any time you need to read from the file, the accessor object can use SetFilePointerEx() before calling ReadFile(). You can then pass the accessor class to the constructor of whatever objects you create when you read the file. The objects then use the accessor class to read the data from the file. Then it returns the data to the object's constructor which parses it into object data.
If, later down the line, you're able to compile to 64-bit, you can just change (or extend) the accessor class to read from memory instead.
As for limiting the amount of RAM used by the process.. that's mostly a matter of making sure that
A) you don't have memory leaks (especially obscene ones) and
B) destroying objects you don't need at the very moment. Even if you will need it later down the line but the data won't change... just destroy the object. Then recreate it later when you do need it, allowing it to re-read the data from the file.
What is the best way to generate a unique hardware ID on Microsoft Windows with C++ that is not easily spoofable (with for example changing the MAC Address)?
Windows stores a unique Guid per machine in the registry at:
HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\MachineGuid
This used to be the CPU serial number but today there are many types of motherboards and this factor is not accurate. MAC address can be easily forged. That leaves us with the internal hard drive serial number. See also: http://www.codeproject.com/Articles/319181/Haephrati-Searching-for-a-reliable-Hardware-ID
There are a variety of "tricks", but the only real "physical answer" is "no, there is no solution".
A "machine" is nothing more than a passive bus with some hardware around.
Although each piece of iron can provide a somehow usable identifier, every piece of iron can be replaced by a user for whatever bad or good reason you can never be fully aware of (so if you base your functionality on this, you create problems to your user, and hence -as a consequence- to yourself every time an hardware have to be replaced / reinitialized / reconfigured etc. etc.).
Now, if your problem is identify a machine in a context where many machines have to inter-operate together, this is a role well played by MAC or IP addresses or Hostnames. But be prepared to the idea that they are not necessarily constant on long time-period (so avoid to hard-code them - instead "discover then" upon any of your start-up)
If your problem is -instead- identify a software instance or a licence, you have probably better to concentrate on another kind of solution: you sell licences to "users" (it is the user that has the money, not his computer!), not to their "machines" (that users must be free to change whenever they need/like without your permission, since you din't licence the hardware or the OS...), hence your problem is not to identify a machine, but a USER (consider that a same machine can be a host for many user and that a same user can work on a variety of machines ..., you cannot assume/impose a 1:1 relation, without running into some kind of problems sooner or later, when this idiom ifs found to no more fit).
The idea should be to register the users in a somewhat reachable site, give them keys you generate, and check that a same user/key pair is not con-temporarily used more than an agreed number of times under a given time period. When violations exceed, or keys becomes old, just block and wait for the user to renew.
As you can see, the answer mostly depends on the reason behind your question, more than from the question itself.
There are various IDs assigned to hardware that can be read and combined to form a machine key. For example, you could get the ID of the hard drive where the software is stored, the proc ID, etc. Some of these can be set more easily than others, but part of the strength is in combining multiple pieces together that are not necessarily strong enough by themselves.
Here is a program (also available as DLL) that can read and show your computer/hardware ID: http://www.soft.tahionic.com/download-hdd_id/index.html
Use Win32 System HDS APIs.
Don't read the registry, it has no sense at all.