email map validation in firestore - regex

So I know that email validation is quite a difficult thing to do. I have already written a regex that checks for a valid email adress. The problem is writing the security rule, seeing as I am dealing with a map in cloud firestore. The map looks like this:
email{
work: ""
personal: ""
}
The problem is the fact that I cannot guarantee that a specific value will match the regex. Users should be able to have only a personal email, only a work email or both a personal and work email. All of the situations should result in validated email adresses in firestore.
I currently have the following code, but I can't figure out how to deal with maps like this:
match /organisations/{orgID}/people/{userID} {
allow create: if(request.resource.data.email.matches(^[A-Za-z0-9]{3,}[#]{1}[A-Za-z0-9]{3,}[.]{1}[A-Za-z0-9]{3,}$) == true);
}
Is this doable with just one security rule? If yes, how? If no, how do I manage this another way? I'd rather use security rules over writing a cloud function for this if possible.
I currently have something like this, but I get an error because firebase doesn't seem to recognize the | (OR) operator. Is there any alternative for doing this? I'm tring to test if the email is either valid or null.
match /organisations/{orgID}/people/{userID}{
allow create: if(
request.resource.data.email.work.matches(^[A-Za-z0-9.]{3,}[#]{1}[A-Za-z0-9.]{3,}[.]{1}[A-Za-z0-9.]{3,}$|"")
);
}
Thanks in advance for your help!

You have two distinct problems here. They are not directly related to each other. I will try to address them separately.
The problem is writing the security rule, seeing as I am dealing with a map in cloud firestore.
If you want to use the value of a nested field within map field, you can simply use dot notation to get to it:
request.resource.data.email.work
request.resource.data.email.personal
The problem is the fact that I cannot guarantee that a specific value will match the regex. Users should be able to have only a personal email, only a work email or both a personal and work email.
You will need to write logic to check each map field separately. You can't check all fields of the map at the same time.

Related

Validation with cleaned_data()

I'd like to ask you if you can briefly and in plain English explain to me
how cleaned_data() function validates data ?
Reason for asking is that I'm designing a web app powered by Django
and initially I thought cleaned_data() is smart enough to block user's input that contains potentially harmful characters. Such as ' ; < > and alike. Characters that can be used for SQL injection attacks.
To my surprise, when I deliberately slipped few of those characters into form field, the input made it to database. I was quite shocked.
So then ... what the cleaned_data() function is good for ?
I read about this function in docs, however I couldn't find necessarily answer to this.
cleaned_data is for validated form data. If you have a required CharField, for example, it will validate whether it is present, and whether it has enough characters. If you have an EmailField, then it will validate that it includes an email address.
Take a look at some of the build in form fields for a better idea of what you can do.
It is not intended to prevent XSS or SQL injection. It simply confirms that your form follows basic rules that you have set for it.
You missunderstood cleaned_data. The simplest definition of cleaned_data is something like:
A dict that contains data entered by the user after various validation
(built-in or custom)
Now, that being said, to understand every steps to form validation refer to this link (re-inventing the wheel would be silly since it is greatly explained.)
As for the SQL injection, this is another problem. But again, Django as a built-in way of handling it, this is from the documentation:
By using Django’s querysets, the resulting SQL will be properly
escaped by the underlying database driver. However, Django also gives
developers power to write raw queries or execute custom sql. These
capabilities should be used sparingly and you should always be careful
to properly escape any parameters that the user can control. In
addition, you should exercise caution when using extra() and RawSQL..
I can totally see your confusion, but remember that they are two different things.

False positive results in email validation on mailgun

We are doing some rather rigorous front end testing in our opt in List.
When users actually subscribed to our email list we are thinking of running email validation ( https://documentation.mailgun.com/api-email-validation.html#email-validation ) before we even save it to the database.
Does anyone know whether this built in email validation Will result in false positive?(In other words is it possible that the function will flag agood address as bad?)
Secondly, if we do have a false positive Address in our database: does mail gun in fact check it before trying to send it anyway. Will it get to the destination?
In case anyone else is searching for this....
It is highly unlikely that there will be a false positive. The APIs structure to allow permissive rather than restrictive: This means that you will not get a false positive but it's very likely that A untrue email address will be flagged as valid
You can use MailboxValidator for a second opinion. The demo page is free and you can also sign up for the free API plan if you like.

RESTful API and Foreign key handling for POSTs and PUTs

I'm helping develop a new API for an existing database.
I'm using Python 2.7.3, Django 1.5 and the django-rest-framework 2.2.4 with PostgreSQL 9.1
I need/want good documentation for the API, but I'm shorthanded and I hate writing/maintaining documentation (one of my many flaws).
I need to allow consumers of the API to add new "POS" (points of sale) locations. In the Postgres database, there is a foreign key from pos to pos_location_type. So, here is a simplified table structure.
pos_location_type(
id serial,
description text not null
);
pos(
id serial,
pos_name text not null,
pos_location_type_id int not null references pos_location_type(id)
);
So, to allow them to POST a new pos, they will need to give me a "pos_name" an a valid pos_location_type. So, I've been reading about this stuff all weekend. Lots of debates out there.
How is my API consumers going to know what a pos_location_type is? Or what value to pass here?
It seems like I need to tell them where to get a valid list of pos_locations. Something like:
GET /pos_location/
As a quick note, examples of pos_location_type descriptions might be: ('school', 'park', 'office').
I really like the "Browseability" of of the Django REST Framework, but, it doesn't seem to address this type of thing, and I actually had a very nice chat on IRC with Tom Christie earlier today, and he didn't really have an answer on what to do here (or maybe I never made my question clear).
I've looked at Swagger, and that's a very cool/interesting project, but take a look at their "pet" resource on their demo here. Notice it is pretty similar to what I need to do. To add a new pet, you need to pass a category, which they define as class Category(id: long, name: string). How is the consumer suppose to know what to pass here? What's a valid id? or name?
In Django rest framework, I can define/override what is returned in the OPTION call. I guess I could come up with my own little "system" here and return some information like:
pos-location-url: '/pos_location/'
in the generic form, it would be: {resource}-url: '/path/to/resource_list'
and that would sort of work for the documentation side, but I'm not sure if that's really a nice solution programmatically. What if I change the resources location. That would mean that my consumers would need to programmatically make and OPTIONS call for the resource to figure out all of the relations. Maybe not a bad thing, but feels like a little weird.
So, how do people handle this kind of thing?
Final notes: I get the fact that I don't really want a "leaking" abstaction here and have my database peaking thru the API layer, but the fact remains that there is a foreign_key constraint on this existing database and any insert that doesn't have a valid pos_location_type_id is raising an error.
Also, I'm not trying to open up the URI vs. ID debate. Whether the user has to use the pos_location_type_id int value or a URI doesn't matter for this discussion. In either case, they have no idea what to send me.
I've worked with this kind of stuff in the past. I think there is two ways of approaching this problem, the first you already said it, allow an endpoint for users of the API to know what is the id-like value of the pos_location_type. Many API's do this because a person developing from your API is gonna have to read your documentation and will know where to get the pos_location_type values from. End-users should not worry about this, because they will have an interface showing probably a dropdown list of text values.
On the other hand, the way I've also worked this, not very RESTful-like. Let's suppose you have a location in New York, and the POST could be something like:
POST /pos/new_york/
You can handle /pos/(location_name)/ by normalizing the text, then just search on the database for the value or some similarity, if place does not exist then you just create a new one. That in case users can add new places, if not, then the user would have to know what fixed places exist, which again is the first situation we are in.
that way you can avoid pos_location_type in the request data, you could programatically map it to a valid ID.

Parameter not supported by web service

I want to validate an opinion with you.
I have to design a web service that searches into a database of restaurants affiliated to a discount program in a specific country around a given address.
The REST call to such a webservice will look like http://server/search?country=<countryCode>&language=<languageCode>&address=<address>&zipcode=<zipcode>
The problem is that some countries do not have zipcodes or do not have them in the entire country.
Now, what would you do if the user passes such a parameter for a country that does not have zipcodes, but he/she passes a valid address?
Return 400 Bad request.
Simply igonre the zipcode parameter and return results based on the valid address
Return an error message in a specific format (e.g. JSON) stating that zipcodes are not supported for that country
Some colleagues are also favoring the following option
4. Simply return no results. And state in the documentation that the zipcode parameter is not supported. Also we have to create a webservice method which returns what fields should be displayed in the user interface.
What option do you think is best and why?
Thanks!
Well the OpenStreetMap Nomination Server returns results even if you dont know the ZIP Code and you can look at the results anyway. What if the user doesnt know the zip code but wants to find hist object?
I would try to search for that specific object anyway, especially because you said that some countries have zip codes partially.
If you simply return nothing te user doesnt know what went wrong and he wont know what to do.
That would depend on the use case. How easy is it for a user of the API to trigger that case? Is it a severe error which the user really should know how to avoid? Or is it something that is not entirely clear, where a user may know (or think he knows) a zipcode where officially there shouldn't be one? Does it come down to trial and error for the user how to retrieve correct results from your API? Is it a bad enough error that the user needs to be informed about it and that he needs to handle this on his side?
If you place this restriction in your API, consider that it will have to be clearly documented when this case is triggered, every user of the API will have to read and understand that documentation, it needs to be clear how to avoid the problem, it needs to be possible for the user to avoid the problem and every user will have to correctly implement extra code on his side to avoid this problem. Is it possible for the user to easily know which areas have zipcodes and which don't?
I think the mantra of "be flexible in what you accept, strict in what you output" applies...

List of mock valid/invalid Email Addresses for Unit Tests

Does anyone know of a list of email addresses (which don’t need to be real) I can use for an email validation assembly for a unit test?
I’ve been looking for such a list and can’t seem to find one. I’m not looking for real addresses, just ones that fit, and the more things I can throw at the test the better. I’ve got 10 right now, but if there is a list, it would give me a more thorough test.
I believe you were looking for something like this:
List of Valid and Invalid Email Addresses
Check the tests of the Apache Commons EmailValidator class:
EmailTest,
EmailValidatorTest.
The EmailValidatorTest in the Hibernate Validator also contains some address.
I like to use the set in this page on email validating regexes because the addresses contain what they're testing inside the email address.
Here is a set of test emails that Dominic Sayers uses to test his isEmail validator:
http://code.iamcal.com/php/rfc822/tests/
For more on isEmail:
http://isemail.info/about