I have a MMORPG server in C++, I've never done scripting before and from my point of view I think it would be degrading to the overall performance of the server if I parse scripts on the go (I haven't tested though), but I would like to have such functionality.
What good scripting techniques for multi-threaded environments that you would suggest/use? A book or an article would be nice too, preferably related to C++ but I don't mind other languages.
Thanks.
I believe the majority of commonly used scripting languages perform parsing as a separate step to execution, so that wouldn't be a significant performance cost. Usually they compile to some kind of bytecode format (Python, Lua and Perl all do this for example), and often that format can be serialised and loaded directly from disk.
There are implementations of scripting languages that compile to native code. For example, you could try javascript and Google's v8 engine, which (as far as I'm aware) compiles everything to native code before execution.
v8 is of course used in Chrome, which is a multi-process environment, so I would imagine it would work perfectly well in a multi-threaded environment (I can't claim personal experience of that though).
There are also JIT compilers for languages that are typically compiled to bytecode (for example, Psyco for python, and LuaJit for Lua). These are often not in sync with the latest version of the main language distribution though.
I think you want to check out Node.js.
It is a high performance multi threaded engine built on top of Google's V8 engine. It's extremely fast and built to be for scaling to huge levels.
Related
My team and I are developing an application that makes use of Flink.
The data will be processed using a computationally-heavy numerical algorithm.
In order to optimize it as much as possible, I would like to write this algorithm in C/C++ rather than in Java.
The question is: is it possible to use C/C++ code within Flink? Perhaps by wrapping it into a Java library?
I've never tested this case in particular. In general, you can always use native code from Java using JNI, the Java Native Interface.
The idea would be to have a Java facade expose your native code and use these methods in your computation graph defined with Flink in Java (or other JVM languages, like Scala). You would have to make both Java and native libraries available on all involved nodes to make this work. If you have an Hadoop cluster you can leverage YARN to ship files along with your job (docs here, see --yarn-ship CLI option).
I would suggest you test this incrementally, with a very small native function exposed. Also, don't underestimate Java's capabilities in terms of performance: with some well thought programming and leveraging JIT and other runtime optimizations, long running processes can enjoy even better performances than similar native code with unmanaged memory.
Keep in mind that this of course resorting to native code will mean restricting the portability of your code to the platforms for which you'll compile your libraries.
I'm designing a simple interpreted language for testing real-time embedded systems. The control flow is severely restricted to provide strong static guarantees on what the scripts will do + how long they will run. For example, you can only branch on constant conditionals or loop over fixed ranges.
There is a large existing codebase in C++ with relevant models and IO libs, so this language must be able to call into C++. The systems under test have hard timing requirements, so we can't tolerate much jitter in the test framework. Our past solution was a custom DSL embedded in the C++ runtime, but we ended up re-inventing too many wheels (parser, linter, interactive interpreter, etc..) to achieve the static guarantees we need.
Haskell's facilities for crafting embedded DSLs with these guarantees are extremely appealing to me, but I'm stuck in determining how to embed it into the soft real-time C++ runtime. Any ideas? Pointers to any libraries / existing projects would be greatly appreciated!
Sounds like the path of least resistance would be an EDSL that generates C++. This way, you don't have to worry about the potential mismatch between soft real-time and the GHC RTS.
You might look at how other EDSLs that generate PLs are implemented:
HJScript uses a free monad approach to embedding JS.
JMacro uses more of an external DSL approach but embedded via TH. Wouldn't be my choice.
Instead of generating strings of C++ code, it's nice to have a data structure. Unfortunately, there doesn't seem to be a package available for C++. However, you could take a look at language-c — perhaps extend that or build your own. You might even consider generating C and using the C to C++ interop provided by those languages.
I'd probably dissuade you from looking at the design of Cryptol or Cogent as these are fully-fledged programming languages (that you have indicated you are inclined to steer away from).
I have read a few articles in the cross-platform tag. However, as I'm starting a fresh application (mostly a terminal/console app), I'm wondering about the easiest way to make it cross-platform (i.e. working for Linux, Mac OS X, and Windows). I have thought about the following:
adding various macro/tags in my code to build different binary executables for each operating system
use Qt platform to develop a cross-functional app (although the GUI and platform component would add more development time as I'm not familiar with Qt)
Your thoughts? Thanks in advance for your contribution!
Edit: Sounds like there are a lot of popular responses on Java and Qt. What are the tradeoffs between these two while we're at it?
Do not go the first way. You'll encounter a lot of problems that are already solved for you by numerous tools.
Qt is an excellent choice if you definitely want C++. In fact, it will speed up development even if you aren't familiar with it, as it has excellent documentation and is easy to use. The good part about it is that it isn't just a GUI framework, but also networking, XML, I/O and lots of other stuff you'll probably need.
If not necessary C++, I'd go with Java. C++ is far too low level language for most applications. Debugging memory management and corrupt stacks can be a nightmare.
To your edited question:
The obvious one: Java has garbage collection, C++ doesn't. It means no memory leaks in Java (unless you count possible bugs in JVM), no need to worry about dangling pointers and such.
Another obvious one: it is extremely easy to use platform-dependent code in C++ using #ifdefs. In Java it is a real pain. There is JNI but it isn't easy to use at all.
Java has very extensive support of exceptions. While C++ has exceptions too, Qt doesn't use them, and some things that generate exceptions in Java will leave you with corrupt memory and crashes in C++ (think buffer overflows).
"Write once, run everywhere." Recompiling C++ program for many platforms can be daunting. Java programs don't need to be recompiled.
It is open to debate, but I think Java has more extensive and well-defined library. The abstraction level is generally higher, the interfaces are cleaner. And it supports more useful things, like XML schemas and such. I can't think of a feature that is present in Qt, but absent in Java. Maybe multimedia or something, I'm not sure.
Both languages are very fast nowadays, so performance is usually not an issue, but Java can be a real memory hog. Not extremely important on modern hardware too, but still.
The least obvious one: C++ can be more portable than Java. One example is FreeBSD OS which had very poor support for Java some time ago (don't know if it is still the case). C++/Qt works perfectly there. If you plan on supporting a wide range of Unix systems, C++ may be a better choice.
Use Java. As much bashing as it gets/used to get, it's the best thing to get stuff working across any platform. Sure, you will still need to handle external OS related functions you may be using, but it's much better than using anything else.
Apart from Java, there are a few things you can run on the JVM - JRuby, Jython, Scala come to mind.
You could also write with the scripting languages directly( Ruby, Python, etc ).
C/C++ is best left for applications that demand complete memory control and high controllability.
I'd go with the QT (or some other framework) option. If you went with the first you'd find it considerably harder. After all, you have to know what to put into the various conditionally compiled sections for all the platforms you're targeting.
I would suggest using a technology designed for cross-platform application development. Here are two technologies I know of that -- as long as you read the documentation and use the features properly -- you can build the application to run on all 3 platforms:
Java
XULRunner (Mozilla's Development Platform)
Of course, there is always the web. I mostly use web applications not just for their portability, but also because they run on my Windows PC, my Ubuntu computer, and my Mac.
We mainly build web applications because the web is the future. Local applications are viewed in my organization as mostly outdated, unless there is of course some feature or technology the web doesn't yet support that holds that application back from being fully web-based.
I would also suggest Github's electron which allows to build cross platform desktop applications using NodeJs and the Google's Chromium. The only drawback for this method is that an electron application run much slower than a native C++ application due to the abstraction layers between Javascript and native C++.
If you're making a console app, you should be able to use the same source for all three platforms if you stick to the functions defined in the POSIX libraries. Setting up your build environment is the most complicated part, especially if you want to be able to build for multiple platforms out of the same source tree.
I'd say if you really want to use C++, QT is the easiest way for cross-platform application, I found myself using QT when I need an UI even though QT has a large set of library which makes pretty much everything easier in C++.
If you don't want to use QT then you need a good design and a lot of abstraction to make cross-platfform application.
However I'm using more and more Python bindinq to QT for medium size application.
If you are working on a console application and you know a bit of python, you might find Python scripting much more comfortable than C++. It keeps the time comsuming stuff away to be able to focus on your application.
Performance considerations aside, is there any known way to take existing C, C++, or Objective C code and run it directly in the browser? For example, a compiler that converts all the code into some interpreted language that can be run in the browser. Like Javascript, or Actionscript and the Flash player, or I suppose Java and the JVM.
I recognize there are higher level languages like Haxe that can be compiled to different targets. And on the other side there are projects like Cappuccino and GWT that attempt to make Javascript development more like traditional desktop development.
But I was wondering if you had an application that worked on a desktop or an existing code base done in C, C++, or Objective C could it easily be converted to a web based application?
Is there work being done on this front? Is there any practical reason to do this? Basically turn the browser into the OS?
Beside the performance issues, and the entrenchment of OS vendors, are there any technical reasons this couldn't be accomplished? Could this kind of C like code be shoehorned into a virtual machine hosted in a browser?
Google's Native Client (NaCl) uses a tweaked compiler to create x86 object code that can be verified by the browser and run in a sandbox, without a major performance hit - pretty cool stuff. They've compiled Quake under it.
This Matasano article has a good run-down on how it works.
Here is a C compiler which targets a number of other languages, including Javascript:
http://cowlark.com/clue/
Not sure what state it's in - last I spoke to the author, it handled pure C89 (subject to the limitations of the compiler frontend). AFAIK there are no plans for it ever to support a GUI.
I was wondering if you had an application that worked on a desktop or an existing code base done in C, C++, or Objective C could it easily be converted to a web based application?
That's sort of what Silverlight is for (C# rather than Objective-C, of course), since it makes the .NET runtime available. Porting a desktop app is usually as much about the GUI as it is about the language - if you have a Cocoa app and you want to port it to another environment (whether that's a browser or Windows), then you'd need more than just an Objective-C cross-compiler, you need the Application Kit and so on. WINE being a notable counter-example, it's pretty rare for these OS-specific libraries to be available at all on other platforms, let alone efficiently. And even where they are available, there are look-and-feel and usability problems when the conventions of one UI are bolted on top of another. So people tend to either use portable frameworks to start with, or else completely rewrite the presentation layer of the app.
Basically turn the browser into the OS?
There are several projects underway to turn the browser into a fully-featured environment for applications (not sure whether or not this is what you mean by "OS"). Flash and AIR, Silverlight, HTML 5. None of them plan to provide C as a programming language, as far as I'm aware.
Emscripten allows you to compile your code into javascript, which is then platform and browser independent.
I think the closest thing you are looking for is Google Native Client. It is still in early development stages though.
You may be interested in LLVM, the Low Level Virtual Machine. It would be possible to implement an LLVM inside a Java applet, Flash applet, or even in Javascript (I wouldn't be surprised if somebody hasn't already done some or all of the above).
Converting an existing application is a completely different kettle of fish, however. The paradigms of user interaction are so completely different between a "desktop" app and a "browser" app that a lot of it will have to be redesigned before a port is reasonable.
Check out Adobe Alchemy (formerly known as FlaCC), which uses LLVM to compile C/C++ to Flash.
This is possible with an ActiveX control, but this works only in microsoft internet explorer.
http://code.google.com/p/cibyl/wiki/Cibyl can make Java sources, so you could compile that for the Java plugin in a browser. Given that Java plugins are much less common nowadays though, you may be better off with a solution that compiles to Javascript.
It seems to me that the major challenges are not related to the language being used. I suspect C would be a very difficult language to implement in JavaScript, but it is possible.
It just seems like a bad idea to me.
First off, I would not write a desktop application in C, much less a web application. Second, web applications require a completely different architecture than desktop applications. Simply cross compiling a desktop application will not make it a web application. If it is portability you are looking for, I suggest using a high-level language targeting the JVM.
Maybe you should consider http://ideone.com for compiling c++ in the browser? You can also compile or interpret other languages, I personally use it rather for more exotic languages - I have c and c++ on my PC :)
I'm a game's developer and am currently in the processing of writing a cross-platform, multi-threaded engine for our company. Arguably, one of the most powerful tools in a game engine is its scripting system, hence I'm on the hunt for a new scripting language to integrate into our engine (currently using a relatively basic in-house engine).
Key features for the desired scripting system (in order of importance) are:
Performance - MUST be fast to call & update scripts
Cross platform - Needs to be relatively easy to port to multiple platforms (don't mind a bit of work, but should only take a few days to port to each platform)
Offline compilation - Being able to pre-parse the script code offline is almost essential (helps with file sizes and load times)
Ability to integrate well with c++ - Should be able to support OO code within the language, and integrate this functionality with c++
Multi-threaded - not required, but desired. Would be best to be able to run separate instances of it on multiple threads that don't interfere with each other (i.e. no globals within the underlying code that need to be altered while running). Critical Section and Mutex based solutions need not apply.
I've so far had experience integrating/using Lua, Squirrel (OO language, based on Lua) and have written an ActionScript 2 virtual machine.
So, what scripting system do you recommend that fits the above criteria? (And if possible, could you also post or link to any comparisons to other scripting languages that you may have)
Thanks,
Grant
Lua has the advantage of being time-tested by a number of big-name video game developers and a good base of knowledgeable developers thanks to Blizzard-Activision's adoption of it as the primary platform for developing World of Warcraft add-ins.
Lua is a very good match for your needs. I'll take them in the same order.
Lua is one of the fastest scripting languages. It's fast to compile and fast to run.
Lua compiles on any platform with an ANSI C compiler, which afaik includes all gaming platforms.
Lua can be pre-compiled, but as a very dynamic languages most errors are only detectable at runtime. Also precompiled code (as bytecode) is often larger in terms of size than source code.
There are many Lua/C++ binding tools.
It doesn't support multi-threading (you cannot access a single instance of the interpreter from multiple threads), but you can have several instances of the interpreter, one per thread, or even one per game object.
Lua have been used in video-game industry for years. Lightweight and efficient.
That being said, ChaiScript and Falcon are good candidates matching your needs and with higher level language than Lua but with less history and community support.
Lua
Boost Python
SWIG
We've had good luck with Squirrel so far. Lua is so popular it's on its way to becoming a standard.
I recommend you worry more about memory than speed. Most scripting languages are "fast enough" and if they get slow you can always push some of that functionality back down into C++. Many of them burn through lots of memory, though, and on a console memory is an even more scarce resource than CPU time. Unbounded memory consumption will crash you eventually, and if you have to allocate 4MB just for the interpreter, that's like having to throw 30 textures out the window to make room.
Lua, and then LuaJIT for extra blaziness!
just don't expect too much from automatic C++ binding libraries, most are slow and restrictive. better do your own binding for your own objects.
as for concurrency, either LuaLanes, or roll your own. if your C++ program is already multithreaded, just call separate LuaStates from each thread, and use your own C++ shared structures as communications channels if needed.
as you might already know, the most often repeated answer in Lua is 'roll your own', and it's often the best advice! except when it's about bindings to common C/C++ libraries, in that case it's quite probable there's already one.
If you haven't looked at it yet I would suggest you check out Angelscript.
I have successfully used it in a cross platform environment (Windows and Linux with only a recompile) and it is designed to integrate well with C++ (both objects and code).
It is lightweight and supports multi-threading (in the sense that the question was asked), performs well and compiles to byte code which could be done in advance.
Start with Python.
If you can prove that you need more speed, then look at Stackless Python. That's what EVE Online uses for their game.
JavaScript may be a reasonable option, because of the mountains of effort that have gone into optimizing the various implementations for use in web-browsers.
These come to mind:
Lua
Python with boost::python
MzScheme or Guile
Ruby with SWIG