I was going through one code on net.
I did not understand following logic. This code works and works really fast.
for (int i = 0; i < typo_word_vec.size(); i++)
{
float each_typo_word_len = (float)typo_word_vec[i].size();
int start_range = each_typo_word_len - floor((each_typo_word_len / lower_bound_word_size) * each_typo_word_len) - 1;
if (start_range < 1)
start_range = 1;
int end_range = each_typo_word_len + ceil((each_typo_word_len / upper_bound_word_size) * each_typo_word_len) + 1;
if (end_range > src_word_max_len)
end_range = src_word_max_len - 1;
call_get_dist(i, start_range, end_range);
}
But I do not understand what is the logic behind using start_range and end_range What underlying algorithm or theory is used here.
You really should have posted a few more lines - we definitely need to check the whole code to understand something.
As I understand it, the 'source' words are ordered by size. The 'candidate' words may be shorter or longer than their potential match. That is what start_range and end_range are used for.
Though I have a hard time figuring why the author doesn't use
start_range = 0;
end_range = src_word_max_len;
EDIT:
ok, this is just an optimization from his part (quoting readme.txt):
I solved this using pythonand php first, however, my solution were continuously rejected since it takes too much time to solve it (my guess). In the "cpp" directory, I uploaded my solution with c++ using STL, and finally accepted (The basic idea of algorithm is almost the same: prunning the scan range of source files) Currently, I plan to try this problem using other language such as java next time. The statement of the problem can be found here: http://www.facebook.com/careers/puzzles.php?puzzle_id=17
He just arbitrarily defines a range large enough to have a high enough probability of finding the right matching word in it.
Related
I understand that this is a pretty math-y question, but how do programs get square roots? From what I've read, this is something that is usually native to the cpu of a device, but I need to be able to do it, probably in c++ (although that's irrelevant).
The reason I need to know about this specifically is that I have an intranet server and I am getting started with crowdsourcing. For this, I am going to start with finding a lot of digits of a certain square root, like sqrt(17) or something.
This is the extent of what python provides is just math.sqrt()
I am going to make a client that can work with other identical clients, so I need complete control over the processes of the math. Heck, this question might not even have an answer, but thanks for your help anyway.
[edit]
I got it working, this is the 'final' product of it: (many thanks to #djhaskin987)
def square_root(number):
old_guess = 1
guess = 2
guesssquared = 0
while round(guesssquared, 10) != round(number, 10):
old_guess = guess
guess = ((number / guess) + guess ) / 2
print(guess)
guesssquared = guess * guess
return guess
solution = square_root(7) #finds square root of 7
print(solution)
Computers use a method that people have actually been using since babylonian times:
def square_root(number):
old_guess = 1
guess = 2
while old_guess != guess:
old_guess = guess
guess = ((number / guess) + guess ) / 2
return guess
x86 has many sqrt in registry, starting with FSQRT for float.
In general, if your function is too complicated or has no implementation, and is C^\infty ("infinitely" differentiable), you can expand it into a polynom via Taylor expansion. This is extremely common in HPC.
I am new to reinforcement learning. I had recently learned about approximate q learning, or feature-based q learning, in which you describe states by features to save space. I have tried to implement this in a simple grid game. Here, the agent is supposed to learn to not go into a firepit(signaled by an f) and to instead eat up as much dots as possible. Here is the grid used:
...A
.f.f
.f.f
...f
Here A signals the agent's starting location. Now, when implementing, I set up two features. One was 1/((distance to closest dot)^2), and the other was (distance to firepit) + 1. When the agent enters a firepit, the program returns with a reward of -100. If it goes to a non firepit position that was already visited(and thus there is no dot to be eaten), the reward is -50. If it goes to an unvisited dot, the reward is +500. In the above grid, no matter what the initial weights are, the program never learns the correct weight values. Specifically, in the output, the first training session gains a score(how many dots it ate) of 3, but for all other training sessions, the score is just 1 and the weights converge to an incorrect value of -125 for weight 1(distance to firepit) and 25 for weight 2(distance to unvisited dot). Is there something specifically wrong with my code or is my understanding of approximate q learning incorrect?
I have tried to play around with the rewards that the environment is giving and also with the initial weights. None of these have fixed the problem.
Here is the link to the entire program: https://repl.it/repls/WrongCheeryInterface
Here is what is going on in the main loop:
while(points != NUMPOINTS){
bool playerDied = false;
if(!start){
if(!atFirepit()){
r = 0;
if(visited[player.x][player.y] == 0){
points += 1;
r += 500;
}else{
r += -50;
}
}else{
playerDied = true;
r = -100;
}
}
//Update visited
visited[player.x][player.y] = 1;
if(!start){
//This is based off the q learning update formula
pairPoint qAndA = getMaxQAndAction();
double maxQValue = qAndA.q;
double sample = r;
if(!playerDied && points != NUMPOINTS)
sample = r + (gamma2 * maxQValue);
double diff = sample - qVal;
updateWeights(player, diff);
}
// checking end game condition
if(playerDied || points == NUMPOINTS) break;
pairPoint qAndA = getMaxQAndAction();
qVal = qAndA.q;
int bestAction = qAndA.a;
//update player and q value
player.x += dx[bestAction];
player.y += dy[bestAction];
start = false;
}
I would expect that both weights would still be positive, but one of them is negative(the one giving distance to the firepit).
I also expected the program to learn overtime that it is bad to enter a firepit and also bad, but not as bad, to go to an unvisited dot.
Probably not the anwser you want to hear, but:
Have you try to implement the simpler tabular Q-learning before approximate Q-learning? In your setting, with a few states and actions, it will work pefectly. If you are learning, I strongly recommend you to start with the simpler cases in order to get a better understanding/intuition about how Reinforcement Learning works.
Do you know the implications of using approximators instead of learning the exact Q function? In some cases, due to the complexity of the problem (e.g., when the state space is continuous) you should approximate the Q function (or the policy, depending on the algorithm), but this may introduce some convergence problems. Additionally, in you case, you are trying to hand-pick some features, which usually required a depth knowledge of the problem (i.e., environment) and the learning algorithm.
Do you understand the meaning of the hyperparameters alpha and gamma? You can not choose them randomly. Sometimes they are critical to obtain the expected results, not always, depending heavely on the problem and the learning algorithm. In your case, taking a look to the convergence curve of you weights, it's pretty clear that you are using a value of alpha too high. As you pointed out, after the first training session your weigths remain constant.
Therefore, practical recommendations:
Be sure to solve your grid game using a tabular Q-learning algorithm before trying more complex things.
Experiment with different values of alpha, gamma and rewards.
Read more in depth about approximated RL. A very good and accesible book (starting from zero knowledge) is the classical Sutton and Barto's book: Reinforcement Learning: An Introduction, which you can obtain for free and was updated in 2018.
I am new to Python, coming from MATLAB, and long ago from C. I have written a script in MATLAB which simulates sediment transport in rivers as a Markov Process. The code randomly places circles of a random diameter within a rectangular area of a specified dimension. The circles are non-uniform is size, drawn randomly from a specified range of sizes. I do not know how many times I will step through the circle placement operation so I use a while loop to complete the process. In an attempt to be more community oriented, I am translating the MATLAB script to Python. I used the online tool OMPC to get started, and have been working through it manually from the auto-translated version (was not that helpful, which is not surprising). To debug the code as I go, I use the
MATLAB generated results to generally compare and contrast against results in Python. It seems clear to me that I have declared variables in a way that introduces problems as calculations proceed in the script. Here are two examples of consistent problems between different instances of code execution. First, the code generated what I think are arrays within arrays because the script is returning results which look like:
array([[ True]
[False]], dtype=bool)
This result was generated for the following code snippet at the overlap_logix operation:
CenterCoord_Array = np.asarray(CenterCoordinates)
Diameter_Array = np.asarray(Diameter)
dist_check = ((CenterCoord_Array[:,0] - x_Center) ** 2 + (CenterCoord_Array[:,1] - y_Center) ** 2) ** 0.5
radius_check = (Diameter_Array / 2) + radius
radius_check_update = np.reshape(radius_check,(len(radius_check),1))
radius_overlap = (radius_check_update >= dist_check)
# Now actually check the overalp condition.
if np.sum([radius_overlap]) == 0:
# The new circle does not overlap so proceed.
newCircle_Found = 1
debug_value = 2
elif np.sum([radius_overlap]) == 1:
# The new circle overlaps with one other circle
overlap = np.arange(0,len(radius_overlap), dtype=int)
overlap_update = np.reshape(overlap,(len(overlap),1))
overlap_logix = (radius_overlap == 1)
idx_true = overlap_update[overlap_logix]
radius = dist_check(idx_true,1) - (Diameter(idx_true,1) / 2)
A similar result for the same run was produced for variables:
radius_check_update
radius_overlap
overlap_update
Here is the same code snippet for the working MATLAB version (as requested):
distcheck = ((Circles.CenterCoordinates(1,:)-x_Center).^2 + (Circles.CenterCoordinates(2,:)-y_Center).^2).^0.5;
radius_check = (Circles.Diameter ./ 2) + radius;
radius_overlap = (radius_check >= distcheck);
% Now actually check the overalp condition.
if sum(radius_overlap) == 0
% The new circle does not overlap so proceed.
newCircle_Found = 1;
debug_value = 2;
elseif sum(radius_overlap) == 1
% The new circle overlaps with one other circle
temp = 1:size(radius_overlap,2);
idx_true = temp(radius_overlap == 1);
radius = distcheck(1,idx_true) - (Circles.Diameter(1,idx_true)/2);
In the Python version I have created arrays from lists to more easily operate on the contents (the first two lines of the code snippet). The array within array result and creating arrays to access data suggests to me that I have incorrectly declared variable types, but I am not sure. Furthermore, some variables have a size, for example, (2L,) (the numerical dimension will change as circles are placed) where there is no second dimension. This produces obvious problems when I try to use the array in an operation with another array with a size (2L,1L). Because of these problems I started reshaping arrays, and then I stopped because I decided these were hacks because I had declared one, or more than one variable incorrectly. Second, for the same run I encountered the following error:
TypeError: 'numpy.ndarray' object is not callable
for the operation:
radius = dist_check(idx_true,1) - (Diameter(idx_true,1) / 2)
which occurs at the bottom of the above code snippet. I have posted the entire script at the following link because it is probably more useful to execute the script for oneself:
https://github.com/smchartrand/MarkovProcess_Bedload
I have set-up the code to run with some initial parameter values so decisions do not need to be made; these parameter values produce the expected results in the MATLAB-based script, which look something like this when plotted:
So, I seem to specifically be having issues with operations on lines 151-165, depending on the test value np.sum([radius_overlap]) and I think it is because I incorrectly declared variable types, but I am really not sure. I can say with confidence that the Python version and the MATLAB version are consistent in output through the first step of the while loop, and code line 127 which is entering the second step of the while loop. Below this point in the code the above documented issues eventually cause the script to crash. Sometimes the script executes to 15% complete, and sometimes it does not make it to 5% - this is due to the random nature of circle placement. I am preparing the code in the Spyder (Python 2.7) IDE and will share the working code publicly as a part of my research. I would greatly appreciate any help that can be offered to identify my mistakes and misapplications of python coding practice.
I believe I have answered my own question, and maybe it will be of use for someone down the road. The main sources of instruction for me can be found at the following three web pages:
Stackoverflow Question 176011
SciPy FAQ
SciPy NumPy for Matlab users
The third web page was very helpful for me coming from MATLAB. Here is the modified and working python code snippet which relates to the original snippet provided above:
dist_check = ((CenterCoordinates[0,:] - x_Center) ** 2 + (CenterCoordinates[1,:] - y_Center) ** 2) ** 0.5
radius_check = (Diameter / 2) + radius
radius_overlap = (radius_check >= dist_check)
# Now actually check the overalp condition.
if np.sum([radius_overlap]) == 0:
# The new circle does not overlap so proceed.
newCircle_Found = 1
debug_value = 2
elif np.sum([radius_overlap]) == 1:
# The new circle overlaps with one other circle
overlap = np.arange(0,len(radius_overlap[0]), dtype=int).reshape(1, len(radius_overlap[0]))
overlap_logix = (radius_overlap == 1)
idx_true = overlap[overlap_logix]
radius = dist_check[idx_true] - (Diameter[0,idx_true] / 2)
In the end it was clear to me that it was more straightforward for this example to use numpy arrays vs. lists to store results for each iteration of filling the rectangular area. For the corrected code snippet this means I initialized the variables:
CenterCoordinates, and
Diameter
as numpy arrays whereas I initialized them as lists in the posted question. This made a few mathematical operations more straightforward. I was also incorrectly indexing into variables with parentheses () as opposed to the correct method using brackets []. Here is an example of a correction I made which helped the code execute as envisioned:
Incorrect: radius = dist_check(idx_true,1) - (Diameter(idx_true,1) / 2)
Correct: radius = dist_check[idx_true] - (Diameter[0,idx_true] / 2)
This example also shows that I had issues with array dimensions which I corrected variable by variable. I am still not sure if my working code is the most pythonic or most efficient way to fill a rectangular area in a random fashion, but I have tested it about 100 times with success. The revised and working code can be downloaded here:
Working Python Script to Randomly Fill Rectangular Area with Circles
Here is an image of a final results for a successful run of the working code:
The main lessons for me were (1) numpy arrays are more efficient for repetitive numerical calculations, and (2) dimensionality of arrays which I created were not always what I expected them to be and care must be practiced when establishing arrays. Thanks to those who looked at my question and asked for clarification.
I have a PID controller working in simulink, but I want to pass it to C++ code. I found how to make a PID with code, something like this:
error = input - refeed;
iError += error * sampleTime;
dError = (error - lastError)/ sampleTime;
//PID Function
output = Kp * error + Ki * iError + Kd * dError;
refeed = output;
lastError = error;
But, that's the only clear thing I got in my research.
I need to know what's the next step, I have the transfer function discretized but I'm not sure about what should I do with the "z" parameters, the times, ...
Is it possible to pass manually a PID controller to C++? How?
The Temperature Control Lab passes a PID output from Python to an Arduino that runs C++ code through a serial USB interface. It is easier to plot values with Python than C++ if you can create an interface for your application. GitHub source code is here.
For the digital control systems, you need to sample the data and execute the controller at every sampling time. z-transform converts the continuous system to the discrete system.
For exampl, if your sampling time is '1', you can express a simple time-series model as below,
y(t) = a1*u(t-1) + a2*u(t-2)
--> y(t) = a1*z^-1*u(t) + a2*z^-2*u(t)
--> y(t) = A(z)u(t), where A(z) = a1*z^-1 + a2*z^-2
a1, a2 = FIR coefficients
However, this time-shift operator 'z^-1' does not appear in your code. It is implicitly expressed with your sampling-time and FOR or DO loop depending on the language that you are using.
Please see the python code for velocity form of PID controller. Velocity form is a little bit easier to implement because you don't worry about the additional logic for the anti-reset windup.
for i in range(1, ns): #ns = simulation time
# PID Velocity form
e[i] = sp[i] - pv[i]
P[i] = Kc * (e[i] - e[i-1])
I[i] = Kc*delta_t/tauI * (e[i])
D[i] = Kc*tauD/delta_t * (pv[i] - 2*(pv[i-1]) + pv[i-2])
op[i] = op[i-1] + P[i] + I[i] + D[i]
if op[i] < oplo or op[i] > ophi:
# clip output
op[i] = max(oplo,min(ophi,op[i]))
You can also find an example of a PID controller using a GEKKO package in the following link.
https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization
Yes it is possible. Have you considered using someone else's code? Or do you want to write it yourself? If you have no problem using allready written code, check out Github. It has a lot of PID projects. For example PID-controller. It has a usage example and you only have to pass in your p, i and d parameters (which you allready got from Matlab).
Good luck!
Basically, you should send the values somewhere. Reading through the comments, you want to make a plot of the output variable in time, so I guess your best bet (and easier way) is to use gnuplot.
Basically, output the data in a text file, then use gnuplot to display it.
I'm programming a Steinberg VST-Plugin in XCode 4.6.
I've already implemented a Highpass-filter which works correctly. Now I'm trying to do some nonlinear distortion with a quadratic function. After I implemented the few lines below and loaded the plugin into the host, I get immediatly an Output from the plugin - you can hear nothing, but the meter is up high.
I really can't imagine why. The processReplacing function where the math takes place should only be called when playing sound, not when the plugin is loaded. When I remove the few lines of code below, everything is okay and sounds right, so I assume it has nothing to do with the rest of the plugin-code.
The problem takes place in two hosts, so its probably not a VST-bug.
Has anybody ever experienced a similar problem?
Many Thanks,
Fabian
void Exciter::processReplacing(float** inputs, float** outputs, VstInt32 sampleFrames){
for(int i = 0; i < sampleFrames; i++) {
tempsample = inputs[0][i];
//Exciter - Transformation in positive region, quadratic distortion and backscaling
tempsample = tempsample + 1.0f;
tempsample = powf(tempsample, 2.0f);
tempsample = tempsample / 2.0f;
tempsample -= 1.0f;
//Mix-Knob: Dry/Wet ------------------------------------------------
outputs[0][i] = mix*(tempsample) + (1-mix)*inputs[0][i];
EDIT: I added logfile-outputs to each function and it occurs, that the processReplacing function is called permanently, not only when playback is turned on ... But why?
You pretty much answered the question yourself with your edit. processReplacing is called repeatedly. This is part of the VST specification.
VST plug-ins are targeted for real time effects processing. Don't confuse or misinterpret this as lookahead. By real time, I mean inserting the plug-in into a channel and playing an instrument while the DAW is recording. So you can see that in order to mitigate latency, the host is always sending the plug-in an audio buffer (whether it's silence or not).