I am new to RUBY and i need to understand 3 functions - c++

I have been given the 3 functions below. Can anybody please help me to understand these? I am trying to port an application to C++ using Qt, but I don't understand these functions. So please help me!
Thanks in advance.
function 1:
def read_key
puts "read pemkey: \"#{#pkey}\"" if #verbose
File.open(#pkey, 'rb') do |io|
#key = OpenSSL::PKey::RSA.new(io)
end
end
function 2:
def generate_key
puts "generate pemkey to \"#{#pkey_o}\"" if #verbose
#key = OpenSSL::PKey::RSA.generate(KEY_SIZE)
# save key
File.open(#pkey_o, 'wb') do |file|
file << #key.export()
end
end
function 3:
def sign_zip
puts "sign zip" if #verbose
plain = nil
File.open(#zip, 'rb') do |file|
plain = file.read
end
#sig = #key.sign(OpenSSL::Digest::SHA1.new, plain)
end

There are probably two things about the above code that are confusing you, which if clarified, will help understand it.
First, #verbose and #key are instance variables, what a C++ programmer might call "member variables." The "if #verbose" following the puts statement literally means only do the puts if #verbose is true. #verbose never needs to be declared a bool--you just start using it. If it's never initialized, it's "nil" which evaluates to false.
Second, the do/end parts are code blocks. Many Ruby methods take a code block and execute it with a variable declared in those pipe characters. An example would be "array.each do |s| puts s; end" which might look like "for(int i = 0; i < array.size(); ++i) { s = array[i]; puts(s); }" in C++. For File.open, |io| is the file instance opened, and "read" is one of its methods.

These are all methods. #{#pkey_o} is string interpolation, substituting in the contents of an instance variable (called pkey_o; Ruby instance variables begin with # and class variables – unused here – begin with ##).
File.open(#pkey, 'rb') do |io|
#key = OpenSSL::PKey::RSA.new(io)
end
That opens the file whose name is stored in #pkey, stores the file handle in io (a block-local variable) and uses that with OpenSSL::PKey::RSA.new, whose result is stored in #key. Finally, it closes the file handle when the block is finished (at the end) whether or not it is a successful exit or an error case (in which case an exception would be thrown, but it would still be thrown). When translating this to C++, use of the RAII pattern is entirely reasonable (if you were going to Java, I'd say to use try/finally).

Related

How can I generate code from file at compile time using a macro?

I have a CSV file that looks like this:
CountryCode,CountryName
AD,Andorra
AE,United Arab Emirates
AF,Afghanistan
AG,Antigua and Barbuda
// -- snip -- //
and a class that looks like this:
module OpenData
class Country
def initialize(#code : String, #name : String)
end
end
end
and I want to have a class variable within the module automatically loaded at compile time like this:
module OpenData
##countries : Array(Country) = {{ run "./sources/country_codes.cr" }}
end
I tried to use the "run" macro above with the following code:
require "csv"
require "./country"
content = File.read "#{__DIR__}/country-codes.csv"
result = [] of OpenData::Country
CSV.new(content, headers: true).each do |row|
result.push OpenData::Country.new(row["CountryCode"], row["CountryName"])
end
result
but this results in
##countries : Array(Country) = {{ run "./sources/country_codes.cr" }}
^
Error: class variable '##countries' of OpenData must be Array(OpenData::Country), not Nil
All my other attempts somehow failed due to various reasons, like not being able to call .new within a macro or stuff like that. This is something I regularly do in Elixir and other languages that support macros and is something I would suspect Crystal can also achieve... I'd also take any other way that accomplishes the task at compile time!
Basically there are several more files I want to process this way, and they`re longer/more complex... thanks in advance!
EDIT:
Found the issue. It seems that I have to return a string that includes actual crystal code from the "run" macro. So, the code in the "run" file becomes:
require "csv"
content = File.read "#{__DIR__}/country-codes.csv"
lines = [] of String
CSV.new(content, headers: true).each do |row|
lines << "Country.new(\"#{row["CountryCode"]}\", \"#{row["CountryName"]}\")"
end
puts "[#{lines.join(", ")}]"
and everything works!
You already found your answer, but for completeness, here are the docs, from: https://crystal-lang.org/api/1.2.2/Crystal/Macros.html#run%28filename%2C%2Aargs%29%3AMacroId-instance-method
Compiles and execute a Crystal program and returns its output
as a MacroId.
The file denoted by filename must be a valid Crystal program.
This macro invocation passes args to the program as regular
program arguments. The program must output a valid Crystal expression.
This output is the result of this macro invocation, as a MacroId.
The run macro is useful when the subset of available macro methods
are not enough for your purposes and you need something more powerful.
With run you can read files at compile time, connect to the internet
or to a database.
A simple example:
# read.cr
puts File.read(ARGV[0])
# main.cr
macro read_file_at_compile_time(filename)
{{ run("./read", filename).stringify }}
end
puts read_file_at_compile_time("some_file.txt")
The above generates a program that will have the contents of some_file.txt.
The file, however, is read at compile time and will not be needed at runtime.

Use Scala Iterator to break up large stream (from string) into chunks using a RegEx match, and then operate on those chunks?

I'm currently using a not-very-Scala-like approach to parse large Unix mailbox files. I'm still learning the language and would like to challenge myself to find a better way, however, I do not believe I have a solid grasp on just what can be done with an Iterator and how to effectively use it.
I'm currently using org.apache.james.mime4j, and I use the org.apache.james.mime4j.mboxiterator.MboxIterator to get a java.util.Iterator from a file, as so:
// registers an implementation of a ContentHandler that
// allows me to construct an object representing an email
// using callbacks
val handler: ContentHandler = new MyHandler();
// creates a parser that parses a SINGLE email from a given InputStream
val parser: MimeStreamParser = new MimeStreamParser(configBuilder.build());
// register my handler
parser.setContentHandler(handler);
// Get a java.util.Iterator
val iterator = MboxIterator.fromFile(fileName).build();
// For each email, process it using above Handler
iterator.forEach(p => parser.parse(p.asInputStream(Charsets.UTF_8)))
From my understanding, the Scala Iterator is much more robust, and probably a lot more capable of handling something like this, especially because I won't always be able to fit the full file in memory.
I need to construct my own version of the MboxIterator. I dug through the source for MboxIterator and was able to find a good RegEx pattern to use to determine the beginning of individual email messages with, however, I'm drawing a blank from now on.
I created the RegEx like so:
val MESSAGE_START = Pattern.compile(FromLinePatterns.DEFAULT, Pattern.MULTILINE);
What I want to do (based on what I know so far):
Build a FileInputStream from an MBOX file.
Use Iterator.continually(stream.read()) to read through the stream
Use .takeWhile() to continue to read until the end of the stream
Chunk the Stream using something like MESSAGE_START.matcher(someString).find(), or use it to find the indexes the separate the message
Read the chunks created, or read the bits in between the indexes created
I feel like I should be able to use map(), find(), filter() and collect() to accomplish this, but I'm getting thrown off by the fact that they only give me Ints to work with.
How would I accomplish this?
EDIT:
After doing some more thinking on the subject, I thought of another way to describe what I think I need to do:
I need to keep reading from the stream until I get a string that matches my RegEx
Maybe group the previously read bytes?
Send it off to be processed somewhere
Remove it from the scope somehow so it doesn't get grouped the next time I run into a match
Continue to read the stream until I find the next match.
Profit???
EDIT 2:
I think I'm getting closer. Using a method like this gets me an iterator of iterators. However, there are two issues: 1. Is this a waste of memory? Does this mean everything gets read into memory? 2. I still need to figure out a way to split by the match, but still include it in the iterator returned.
def split[T](iter: Iterator[T])(breakOn: T => Boolean):
Iterator[Iterator[T]] =
new Iterator[Iterator[T]] {
def hasNext = iter.hasNext
def next = {
val cur = iter.takeWhile(!breakOn(_))
iter.dropWhile(breakOn)
cur
}
}.withFilter(l => l.nonEmpty)
If I understand correctly, you want to lazily chunk a large file delimited by a regex recognizable pattern.
You could try to return an Iterator for each request but the correct iterator management would not be trivial.
I'd be inclined to hide all file and iterator management from the client.
class MBox(filePath :String) {
private val file = io.Source.fromFile(filePath)
private val itr = file.getLines().buffered
private val header = "From .+ \\d{4}".r //adjust to taste
def next() :Option[String] =
if (itr.hasNext) {
val sb = new StringBuilder()
sb.append(itr.next() + "\n")
while (itr.hasNext && !header.matches(itr.head))
sb.append(itr.next() + "\n")
Some(sb.mkString)
} else {
file.close()
None
}
}
testing:
val mbox = new MBox("so.txt")
mbox.next()
//res0: Option[String] =
//Some(From MAILER-DAEMON Fri Jul 8 12:08:34 2011
//some text AAA
//some text BBB
//)
mbox.next()
//res1: Option[String] =
//Some(From MAILER-DAEMON Mon Jun 8 12:18:34 2012
//small text
//)
mbox.next()
//res2: Option[String] =
//Some(From MAILER-DAEMON Tue Jan 8 11:18:14 2013
//some text CCC
//some text DDD
//)
mbox.next() //res3: Option[String] = None
There is only one Iterator per open file and only the safe methods are invoked on it. The file text is realized (loaded) only on request and the client gets just what's requested, if available. Instead of all lines in one long String you could return each line as part of a collection, Seq[String], if that's more applicable.
UPDATE: This can be modified for easy iteration.
class MBox(filePath :String) extends Iterator[String] {
private val file = io.Source.fromFile(filePath)
private val itr = file.getLines().buffered
private val header = "From .+ \\d{4}".r //adjust to taste
def next() :String = {
val sb = new StringBuilder()
sb.append(itr.next() + "\n")
while (itr.hasNext && !header.matches(itr.head))
sb.append(itr.next() + "\n")
sb.mkString
}
def hasNext: Boolean =
if (itr.hasNext) true else {file.close(); false}
}
Now you can .foreach(), .map(), .flatMap(), etc. But you can also do dangerous things like .toList which will load the entire file.

variable type tagged additionally with Nil in ensure clause

I wonder why the type of the variable is (String | Nil) and not just String? Is there a way one can make it just String?
def main
text = "hello"
ensure
puts typeof(text) # => (String | Nil)
end
main
https://carc.in/#/r/2w3a
ensure runs after the main body in any case, even if there was an exception raised. Because this could have happend anywhere, it has to be considered that the body of the method hasn't been executed at all if it failed at the first instruction.
Therefore, in the ensure block, all variables are known but it must be assumed that their value can be nil.
If you're sure that text is always set, you don't need to protect that assignment in a rescue/ensure clause.
def main
text = "hello"
begin
# here is the code that might fail
ensure
puts typeof(text) # => String
end
end

How to save console output to string in crystal?

On ruby I can do
require "stringio"
def with_captured_stdout
begin
old_stdout = $stdout
$stdout = StringIO.new('','w')
yield
$stdout.string
ensure
$stdout = old_stdout
end
end
and later call it like
str = with_captured_stdout { Solution.main("Greetings from Javatlacati") }
but on crystal-lang I can refer to global variable $stdout without getting the corresponding error
$global_variables are not supported, use ##class_variables instead
is there any workaround? Thank you in advance.
Currently there's no easy way to capture stdout or stderr in the crystal standard library, however here's a shard which looks to do that: https://github.com/mosop/stdio.

Using regular expressions in python to determine C++ functions and their parameters

So I'm doing something wrong in this python script, but it's becoming convoluted and I'm losing sight of what I'm doing wrong.
I want a script to go through a file, find all the function definitions, and then pull out the name, return type, and parameters of the function, and output a "doxygen" style comment like this:
/******************************************************************************/
/*!
\brief
Main function for the file
\return
The exit code for the program
*/
/******************************************************************************/
But I'm doing something wrong with the regular expression in trying to parse the parameters... Here is the script so far:
import re
import sys
f = open(sys.argv[1])
functions = []
for line in f:
match = re.search(r'([\w]+)\s+([\S]+)\(([\w+\s+\w+])+\)',line)
if line.find("\\fn") < 0:
if match:
returntype = match.group(1)
funcname = match.group(2)
print '/********************************************************************'
print " \\fn " + match.group()
print ''
print ' \\brief'
print ' Function description for ' + funcname
print ''
if len(match.groups()) > 2:
params = []
count = len(match.groups()) - 2
while count > 0:
matchingstring = match.group(count + 2)
if matchingstring.find("void") < 0:
params.append(matchingstring)
count -= 1
for parameter in params:
print " \\param " + parameter
print ' Description of ' + parameter
print ''
print ' \\return'
print ' ' + returntype
print '********************************************************************/'
print ''
Any help would be appreciated. Thanks
The grammar of C++ is far too complex to be handled by simple
regular expressions. You'll need at least a minimal parser.
I've found that for restricted cases, where I'm not concerned
with C++ in general, but only my own style, I can often get away
with a flex based tokenizer and a simple state machine. This
will fail in many cases of legal C++—for starters, of
course, if someone uses the pre-processor to modify the syntax;
but also because < can have different meanings, depending on
what precedes it names a template or not. But it's often
adequate for a specific job.
I've used a PEG parser with great success when trying to do simple format parsing. pyPeg is a very simple implementation of such a parser written in Python.
Example Python code for C++ function parser:
EDIT: Address template parameters. Tested with input from SK-logic and output is correct.
import pyPEG
from pyPEG import parseLine
import re
def symbol(): return re.compile(r"[abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ&*][\w:]+")
def type(): return symbol
def functionName(): return symbol
def templatedType(): return symbol, "<", -1, [templatedType, symbol, ","], ">"
def parameter(): return [templatedType, type], symbol
def template(): return "<", -1, [symbol, template], ">"
def function(): return [type, templatedType], functionName, -1, template, "(", -1, [",", parameter], ")" # -1 -> zero or more repetitions.
sourceCode = "std::string foobar(std::vector<int> &A, std::map<std::string, std::vector<std::string> > &B)"
results = parseLine(sourceCode, function(), [], packrat=True)
When this is executed results is:
([(u'type', [(u'symbol', 'std::string')]), (u'functionName', [(u'symbol', 'foobar')]), (u'parameter', [(u'templatedType', [(u'symbol', 'std::vector'), (u'symbol', 'int')]), (u'symbol', '&A')]), (u'parameter', [(u'templatedType', [(u'symbol', 'std::map'), (u'symbol', 'std::string'), (u'templatedType', [(u'symbol', 'std::vector'), (u'symbol', 'std::string')])]), (u'symbol', '&B')])], '')
C++ cannot really be parsed by a (sane) regular expression: they are a nightmare as soon as nesting is concerned.
There is another concern too, determining when to parse and when not to. A function may be declared:
at file scope
in a namespace
in a class
And the two last can be nested at arbitrary depths.
I would propose to use CLang here. It's a real C++ front-end with a full-featured parser and there are:
a C API, with (notably) an API to the Indexing Library
Python bindings on top of the C API
The C API and Python bindings are far from fully exposing the underlying C++ model, but for a task as simple as listing functions it should be enough.
That said, I would question the usefulness of the project: if the documentation can be generated by a simple parser, then it is redundant with the code. And redundancy is at best, useless, and worst dangerous: it introduces the potential threat of desynchronization...
If the function is tricky enough that its use requires documentation, then a developer, who knows the limitations and al, has to write this documentation.