How to validate an expression in Apache Calcite - apache-calcite

I want to know how to validate a expression in Calcite. I create a schema and add my table into it, then I want to use Validator to validate a SqlNode which is from an expression. Here is the code.
public class DummyTable extends AbstractTable {
private final String tableName;
public DummyTable(String tableName) {
this.tableName = tableName;
}
#Override
public RelDataType getRowType(RelDataTypeFactory typeFactory) {
RelDataTypeFactory.Builder builder = typeFactory.builder();
builder.add("a", SqlTypeName.BIGINT);
builder.add("b", SqlTypeName.BIGINT);
return builder.build();
}
public String getTableName() {
return tableName;
}
}
public class ConditionValidator {
private final String sql;
private final SchemaPlus rootSchema;
private final FrameworkConfig frameworkConfig;
private final RelDataTypeFactory relDataTypeFactory;
private final CalciteCatalogReader catalogReader;
private final SqlOperatorTable sqlOperatorTable;
private final SqlValidator validator;
public ConditionValidator(String sql) {
this.sql = sql;
SchemaPlus rootSchema = Frameworks.createRootSchema(true);
DummyTable testTable = new DummyTable("test_table");
rootSchema.add(testTable.getTableName(), testTable);
this.rootSchema = rootSchema;
this.frameworkConfig = Frameworks.newConfigBuilder()
.parserConfig(SqlParser.config()
.withLex(Lex.MYSQL)
.withConformance(SqlConformanceEnum.DEFAULT))
.defaultSchema(rootSchema)
.operatorTable(SqlStdOperatorTable.instance())
.build();
this.relDataTypeFactory = new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
Properties properties = new Properties();
properties.setProperty(CalciteConnectionProperty.CASE_SENSITIVE.camelName(), "true");
this.catalogReader = new CalciteCatalogReader(
CalciteSchema.from(rootSchema),
CalciteSchema.from(rootSchema).path(rootSchema.getName()),
relDataTypeFactory,
new CalciteConnectionConfigImpl(properties));
this.sqlOperatorTable = SqlOperatorTables.chain(frameworkConfig.getOperatorTable(), catalogReader);
this.validator = SqlValidatorUtil.newValidator(sqlOperatorTable, catalogReader, relDataTypeFactory, frameworkConfig.getSqlValidatorConfig());
}
public SqlNode validate() {
SqlParser sqlParser = SqlParser.create(sql, frameworkConfig.getParserConfig());
SqlNode sqlNode;
try {
sqlNode = sqlParser.parseExpression();
} catch (Exception e) {
throw new RuntimeException(e);
}
SqlNode validate = validator.validate(sqlNode);
return validate;
}
}
And now I can validate a SqlNode from a SQL query, for example, when input sql is "select a, b from test_table where a > 1 and b = 1", validation is ok (if validating a query, you should use the way below to parse the sql:
sqlNode = sqlParser.parseQuery(); // not 'sqlParser.parseExpression();
).
However, when I validate an expression like "a > 1 and b = 1", such an exception occurs: "Column 'a' not found in any table". I assume there maybe something wrong with validate scope, but I can't find the solution. Can someone help me? Thanks a lot!

Related

insert whole list of data into sql database table in xamarin.android

i'm trying to create an application where data in a list must be inserted into a database table at once. I made some research and found out that this is possible using user-defined table types where in c# a datatable is used and passed to a stored procedure that is executed. now my problem is that there are no data tables in Xamarin.Android. so I thought to use a list instead. my idea was to create a list in the application and pass it to the webservice method, and in my webservice method I receive the list and convert it to a datatable then pass it as a parameter to the stored procedure. I wrote the following codes:
in my webservice:
[WebMethod]
public bool insrt_dt(List<Class1> lst)
{
SqlParameter param;
SqlConnection conn = new SqlConnection(new DBConnection().ConnectionString);
DataTable dt = list_to_dt(lst);
SqlCommand cmd = new SqlCommand("Insert_Customers", conn);
cmd.CommandType = CommandType.StoredProcedure;
if (conn.State == System.Data.ConnectionState.Closed)
{
conn.Open();
}
param = new SqlParameter("#tblcustomers", dt);
param.Direction = ParameterDirection.Input;
param.DbType = DbType.String;
cmd.Parameters.Add(param);
cmd.CommandTimeout = 300;
int a=cmd.ExecuteNonQuery();
if (a > 0)
{
return true;
}
else return false;
}
}
Class1:
public class Class1
{
public int id { get; set; }
public string name { get; set; }
public string country { get; set; }
}
in my Xamarin.Android app
protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
// Set our view from the "main" layout resource
SetContentView(Resource.Layout.activity_main);
Button btn = FindViewById<Button>(Resource.Id.button1);
btn.Click += delegate
{
wr.WebService1 ws = new wr.WebService1();
wr.Class1 class1 = new wr.Class1();
List<wr.Class1> lst = new List<wr.Class1>(){
new wr.Class1() { id = 1, name = "hgf", country = "khg" },
new wr.Class1() { id = 2, name = "hgf", country = "khg"} };
ws.insrt_dt(lst);
ws.insrt_dtCompleted += Ws_insrt_dtCompleted;
};
}
private void Ws_insrt_dtCompleted(object sender, wr.insrt_dtCompletedEventArgs e)
{
bool l = e.Result;
if (l == true)
Toast.MakeText(this, "khh", ToastLength.Long).Show();
else
Toast.MakeText(this, "ijo'pioo", ToastLength.Long).Show();
}
}
but I keep getting this error:
Argument 1: cannot convert from 'System.Collections.Generic.List<app_dt.wr.Class1>' to 'app_dt.wr.Class1[]
so I used these lines instead
new wr.Class1() { id = 1, name = "hgf", country = "khg" },
new wr.Class1() { id = 2, name = "hgf", country = "khg"} };
wr.Class1[] class1s = lst.ToArray();
ws.insrt_dt(class1s);
now I don't get an error, but it doesn't work, I mean why does it say that the webservice method input must be an array and I've created it as a list. any suggestions for this?
As i know, Xamarin do not support System.Data.SqlClient. If you want to use the database for the Xamarin android project, you could use the SQLite.NET.
Install the package from the NuGet.
NuGet: sqlite-net-pcl https://www.nuget.org/packages/sqlite-net-pcl/
For the code sample about how to use the database in Xamarin, you could check the link below. https://learn.microsoft.com/en-us/xamarin/get-started/quickstarts/database?pivots=windows
For Xamarin.Android, you could check the sample code in the link below. https://learn.microsoft.com/en-us/xamarin/android/data-cloud/data-access/using-sqlite-orm

XUnit mocked db connection dapper error, 'Object is not set to an instance of an object' when executing sql statement

I am trying to unit test my service layer as advised by #NKosi Here. I am able to do the integration test successfully by implementing the actual factory implementation without mocking anything but can't do the unit test (by mocking IDbConnection and my SQL connection factory class) as Dapper query executing fails with the error 'Object not set to an instance of an object'.
My IDbConnection factory and its implementation is as follow
public interface IDbConnectionFactory
{
IDbConnection CreateConnection();
}
public class ConnectionSetings
{
public string Name { get; set; }
}
public class SqlConnectionFactory : IDbConnectionFactory
{
private readonly ConnectionSetings connectionSettings;
public SqlConnectionFactory(ConnectionSetings connectionSettings)
{
this.connectionSettings = connectionSettings;
}
public IDbConnection CreateConnection()
{
return new SqlConnection(connectionSettings.Name);
}
}
And the XUnit test is as follow
[Fact]
public void Get_RestaurantById_ReturnsRestaurant()
{
//Arrange
var connection = new Mock<IDbConnection>();
var dbConnectionFactory = new Mock<IDbConnectionFactory>();
dbConnectionFactory.Setup(x => x.CreateConnection()).Returns(connection.Object);
//Act
var result = new SqlRestaurantDataCL(dbConnectionFactory.Object).Get(1);
//Assert
result.Name.Equals("Test Name 1");
//Assert.Equal("Test Name 1", result.Name);
}
And the Service Layer is as follow
public class SqlRestaurantDataCL : IRestaurantDataCL
{
private readonly IDbConnectionFactory factory;
public SqlRestaurantDataCL(IDbConnectionFactory factory)
{
this.factory = factory;
}
public Restaurant Get(int id)
{
using (var connection = factory.CreateConnection())
{
var selectSql = #"SELECT * From Restaurants Where Id = #Id";
var restaurant = connection.QuerySingleOrDefault<Restaurant>(selectSql, new
{
id
});
return restaurant;
}
}
}
Following is the error screenshot
Following is the answer to my question if anyone is in similar situation. Before following this solution, I would suggest to read #NKosi comments above and consult #Mikhail's solution Here.
ServiceStack.OrmLite.Sqlite package added to use in memory appraoch
internal class InMemoryDatabase
{
private readonly OrmLiteConnectionFactory dbFactory = new OrmLiteConnectionFactory(":memory:", SqliteDialect.Provider);
public IDbConnection OpenConnection() => this.dbFactory.OpenDbConnection();
public void Insert<T>(IEnumerable<T> items)
{
using (var db = this.OpenConnection())
{
db.CreateTableIfNotExists<T>();
foreach (var item in items)
{
db.Insert(item);
}
}
}
}
Data Access layer is as follow
public IEnumerable<Restaurant> GetAll()
{
using (var connection = factory.CreateConnection())
{
//return connection.Query<Restaurant>("Select * From [dbo].[Restaurants] Order By Name");
return connection.Query<Restaurant>("Select * From Restaurant Order By Name");
}
}
Unit test is as follow
[Fact]
public void Get_RestaurantById_ReturnsRestaurant()
{
//Arrange
var restaurants = new List<Restaurant>
{
new Restaurant { Id = 1, Name = "Test Name 1", Cuisine = CuisineType.None},
new Restaurant { Id = 2, Name = "Test Name 2", Cuisine = CuisineType.French},
new Restaurant { Id = 3, Name = "Test Name 3", Cuisine = CuisineType.German},
new Restaurant { Id = 4, Name = "Test Name 4", Cuisine = CuisineType.Italian},
new Restaurant { Id = 5, Name = "Test Name 5", Cuisine = CuisineType.None}
};
var db = new InMemoryDatabase();
db.Insert(restaurants);
var connection = new Mock<IDbConnection>();
var dbConnectionFactoryMock = new Mock<IDbConnectionFactory>();
dbConnectionFactoryMock.Setup(c => c.CreateConnection()).Returns(db.OpenConnection());
//Act
var result = new SqlRestaurantDataCL(dbConnectionFactoryMock.Object).GetAll();
//Assert
result.Should().BeEquivalentTo(restaurants);
}

Illegal query expression: No hash key condition is found in the query in AWS Query

I have table in AWS mobile hub and I am using the following model for it
public class UserstopcoreDO {
private String _userId;
private String _usertoplevel;
private String _usertopscore;
private String _username;
#DynamoDBHashKey(attributeName = "userId")
#DynamoDBAttribute(attributeName = "userId")
public String getUserId() {
return _userId;
}
public void setUserId(final String _userId) {
this._userId = _userId;
}
#DynamoDBAttribute(attributeName = "usertoplevel")
public String getUsertoplevel() {
return _usertoplevel;
}
#DynamoDBAttribute(attributeName = "username")
public String getUsername() {
return _username;
}
public void setUsername(final String _username) {
this._username = _username;
}
public void setUsertoplevel(final String _usertoplevel) {
this._usertoplevel = _usertoplevel;
}
#DynamoDBIndexHashKey(attributeName = "usertopscore", globalSecondaryIndexName = "usertopscore")
public String getUsertopscore() {
return _usertopscore;
}
public void setUsertopscore(final String _usertopscore) {
this._usertopscore = _usertopscore;
}
}
In the table, I have 1500+ records and now I want to fetch Top 10 records from it so for that I write the below query
final DynamoDBQueryExpression<UserstopcoreDO> queryExpression = new DynamoDBQueryExpression<>();
queryExpression.withLimit(10);
queryExpression.setScanIndexForward(false);
final PaginatedQueryList<UserstopcoreDO> results = mapper.query(UserstopcoreDO.class, queryExpression);
Iterator<UserstopcoreDO> resultsIterator = results.iterator();
if (resultsIterator.hasNext()) {
final UserstopcoreDO item = resultsIterator.next();
try {
Log.d("Item :",item.getUsertopscore());
} catch (final AmazonClientException ex) {
Log.e(LOG_TAG, "Failed deleting item : " + ex.getMessage(), ex);
}
}
But when I run the code it gives me an error
Caused by: java.lang.IllegalArgumentException: Illegal query expression: No hash key condition is found in the query
but in my condition, I did not need any condition because I want to fetch top 10 records instead of one specific record. So how to handle that condition ?
If you want to "query" DynamoDB without specifying all HashKeys, use a Scan instead, i.e. DynamoDBScanExpression. You probably also want to change your "usertopscore" to be a RangeKey instead of a HashKey.
From https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/dynamodbv2/datamodeling/DynamoDBQueryExpression.html every DynamoDBQueryExpression requires all the Hash Keys be set.
Also see boto dynamodb2: Can I query a table using range key only?
Please set the hash key in the query expression. Below is the example of query expression for main table and GSI (need to set the index name).
Querying the main table:-
Set the hash key value of the table.
UserstopcoreDO hashKeyObject = new UserstopcoreDO();
hashKeyObject.setUserId("1");
DynamoDBQueryExpression<UserstopcoreDO> queryExpressionForMainTable = new DynamoDBQueryExpression<UserstopcoreDO>()
.withHashKeyValues(hashKeyObject);
Querying the Index:-
Set the index name and hash key value of the index.
UserstopcoreDO hashIndexKeyObject = new UserstopcoreDO();
hashIndexKeyObject.setUsertoplevel("100");
DynamoDBQueryExpression<UserstopcoreDO> queryExpressionForGsi = new DynamoDBQueryExpression<UserstopcoreDO>()
.withHashKeyValues(hashIndexKeyObject).withIndexName("usertopscore");
GSI attributes in mapper:-
#DynamoDBIndexHashKey(attributeName = "usertoplevel", globalSecondaryIndexName = "usertopscore")
public String getUsertoplevel() {
return _usertoplevel;
}
#DynamoDBIndexRangeKey(attributeName = "usertopscore", globalSecondaryIndexName = "usertopscore")
public String getUsertopscore() {
return _usertopscore;
}

DynamoDBMapper ConditionalCheckFailedException when range-key attribute for GSI is not present in update request

I'm trying to reason about the cause of a ConditionalCheckFailedException I receive when using DynamoDBMapper with a specific save expression and with UPDATE_SKIP_NULL_ATTRIBUTES SaveBehavior.
My schema is as follows:
Member.java
#Data
#DynamoDBTable(tableName = "members")
public class Member implements DDBTable {
private static final String GROUP_GSI_NAME = "group-gsi";
#DynamoDBHashKey
#DynamoDBAutoGeneratedKey
private String memberId;
#DynamoDBVersionAttribute
private Long version;
#DynamoDBIndexHashKey(globalSecondaryIndexName = GROUP_GSI_NAME)
private String groupId;
#DynamoDBAutoGeneratedTimestamp(strategy = DynamoDBAutoGenerateStrategy.CREATE)
#DynamoDBIndexRangeKey(globalSecondaryIndexName = GROUP_GSI_NAME)
private Date joinDate;
#DynamoDBAttribute
private String memberName;
#Override
#DynamoDBIgnore
public String getHashKeyColumnName() {
return "memberId";
}
#Override
#DynamoDBIgnore
public String getHashKeyColumnValue() {
return memberId;
}
}
I use the following class to create/update/get the records in the members table.
DDBModelDAO.java
public class DDBModelDAO<T extends DDBTable> {
private final Class<T> ddbTableClass;
private final AmazonDynamoDB amazonDynamoDB;
private final DynamoDBMapper dynamoDBMapper;
public DDBModelDAO(Class<T> ddbTableClass, AmazonDynamoDB amazonDynamoDB, DynamoDBMapper dynamoDBMapper) {
this.ddbTableClass = ddbTableClass;
this.amazonDynamoDB = amazonDynamoDB;
this.dynamoDBMapper = dynamoDBMapper;
}
public T loadEntry(final String hashKey) {
return dynamoDBMapper.load(ddbTableClass, hashKey);
}
public T createEntry(final T item) {
dynamoDBMapper.save(item, getSaveExpressionForCreate(item));
return item;
}
public T updateEntry(final T item) {
dynamoDBMapper.save(item, getSaveExpressionForUpdate(item),
DynamoDBMapperConfig.SaveBehavior.UPDATE_SKIP_NULL_ATTRIBUTES.config());
return item;
}
private DynamoDBSaveExpression getSaveExpressionForCreate(final T item) {
// No record with the same hash key must be present when creating
return new DynamoDBSaveExpression().withExpectedEntry(item.getHashKeyColumnName(),
new ExpectedAttributeValue(false));
}
private DynamoDBSaveExpression getSaveExpressionForUpdate(final T item) {
// The hash key for the record being updated must be present.
return new DynamoDBSaveExpression().withExpectedEntry(item.getHashKeyColumnName(),
new ExpectedAttributeValue(new AttributeValue(item.getHashKeyColumnValue()))
.withComparisonOperator(ComparisonOperator.EQ)
);
}
}
I wrote a test class to insert and update records into the members table, which is as follows:
public static void main(String[] args) {
DDBTestClient testClient = new DDBTestClient();
AmazonDynamoDB amazonDynamoDB = testClient.buildAmazonDynamoDB();
DynamoDBMapper dynamoDBMapper = testClient.buildDynamoDBMapper(amazonDynamoDB);
DDBModelDAO<Member> memberDAO = new DDBModelDAO<>(Member.class, amazonDynamoDB, dynamoDBMapper);
DDBModelDAO<Group> groupDAO = new DDBModelDAO<>(Group.class, amazonDynamoDB, dynamoDBMapper);
try {
// Create a group
Group groupToCreate = new Group();
groupToCreate.setGroupName("group-0");
Group createdGroup = groupDAO.createEntry(groupToCreate);
System.out.println("Created group: " + createdGroup);
Thread.sleep(3000);
// Create a member for the group
Member memberToCreate = new Member();
memberToCreate.setGroupId(createdGroup.getGroupId());
memberToCreate.setMemberName("member-0");
Member createdMember = memberDAO.createEntry(memberToCreate);
System.out.println("Created member: " + createdMember);
Thread.sleep(3000);
// Update member name
createdMember.setMemberName("member-updated-0");
createdMember.setGroupId(null);
//createdMember.setJoinDate(null); // <---- Causes ConditionalCheckFailedException
memberDAO.updateEntry(createdMember);
System.out.println("Updated member");
} catch (Exception exception) {
System.out.println(exception.getMessage());
}
}
As can be seen above, if I do not pass a valid value for joinDate(which happens to be the range-key for the groups GSI), in the updateEntry call, DynamoDB returns a ConditionalCheckFailedException. This is the case, even when I use a save behavior of UPDATE_SKIP_NULL_ATTRIBUTES, as can be seen in DDBModelDAO.java.
Can someone help me understand, why I'm required to send the range-key attribute for the GSI, for a conditional write to succeed?
Not sure if this answers your question:
Interface DynamoDBAutoGenerator:
"DynamoDBAutoGenerateStrategy.CREATE, instructs to generate when
creating the item. The mapper, determines an item is new, or
overwriting, if it's current value is null. There is a limitation when
performing partial updates using either,
DynamoDBMapperConfig.SaveBehavior.UPDATE_SKIP_NULL_ATTRIBUTES, or DynamoDBMapperConfig.SaveBehavior.APPEND_SET. A new value will only be generated if the mapper is also generating the key."
So the last part is important: "A new value will only be generated if the mapper is also generating the key"
That should explain why you only see the behavior that you are experiencing.
Does this make sense?

ASP.Net MVC: how to write unit test code when working with ValidationAttribute and IClientValidatable

apologized to post bit similar question here. i am bit familiar with asp.net mvc but very new in unit testing. do not think that i know lot just see my reputation in stackoverflow.
i like to know how to write unit test code for IsValid and IEnumerable<ModelClientValidationRule> GetClientValidationRules
here i am pasting my code including my model. so anyone help me to write unit test code for the above two function. i am new in unit testing and working with VS2013 and using VS unit testing framework.
my main problem is how to write unit test code for this function specifically IEnumerable<ModelClientValidationRule> GetClientValidationRules
so here is my full code. anyone who often work with unit test then please see and come with code and suggestion if possible. thanks
Model
public class DateValTest
{
[Display(Name = "Start Date")]
[DataType(DataType.Date), DisplayFormat(DataFormatString = "{0:MM/dd/yyyy}", ApplyFormatInEditMode = true)]
public DateTime? StartDate { get; set; }
[Display(Name = "End Date")]
[DataType(DataType.Date), DisplayFormat(DataFormatString = "{0:MM/dd/yyyy}", ApplyFormatInEditMode = true)]
[DateGreaterThanAttribute(otherPropertyName = "StartDate", ErrorMessage = "End date must be greater than start date")]
public DateTime? EndDate { get; set; }
}
custom validation code
public class DateGreaterThanAttribute : ValidationAttribute, IClientValidatable
{
public string otherPropertyName;
public DateGreaterThanAttribute() { }
public DateGreaterThanAttribute(string otherPropertyName, string errorMessage)
: base(errorMessage)
{
this.otherPropertyName = otherPropertyName;
}
protected override ValidationResult IsValid(object value, ValidationContext validationContext)
{
ValidationResult validationResult = ValidationResult.Success;
try
{
// Using reflection we can get a reference to the other date property, in this example the project start date
var containerType = validationContext.ObjectInstance.GetType();
var field = containerType.GetProperty(this.otherPropertyName);
var extensionValue = field.GetValue(validationContext.ObjectInstance, null);
if(extensionValue==null)
{
//validationResult = new ValidationResult("Start Date is empty");
return validationResult;
}
var datatype = extensionValue.GetType();
//var otherPropertyInfo = validationContext.ObjectInstance.GetType().GetProperty(this.otherPropertyName);
if (field == null)
return new ValidationResult(String.Format("Unknown property: {0}.", otherPropertyName));
// Let's check that otherProperty is of type DateTime as we expect it to be
if ((field.PropertyType == typeof(DateTime) || (field.PropertyType.IsGenericType && field.PropertyType == typeof(Nullable<DateTime>))))
{
DateTime toValidate = (DateTime)value;
DateTime referenceProperty = (DateTime)field.GetValue(validationContext.ObjectInstance, null);
// if the end date is lower than the start date, than the validationResult will be set to false and return
// a properly formatted error message
if (toValidate.CompareTo(referenceProperty) < 1)
{
validationResult = new ValidationResult(ErrorMessageString);
}
}
else
{
validationResult = new ValidationResult("An error occurred while validating the property. OtherProperty is not of type DateTime");
}
}
catch (Exception ex)
{
// Do stuff, i.e. log the exception
// Let it go through the upper levels, something bad happened
throw ex;
}
return validationResult;
}
public IEnumerable<ModelClientValidationRule> GetClientValidationRules(ModelMetadata metadata, ControllerContext context)
{
var rule = new ModelClientValidationRule
{
ErrorMessage = FormatErrorMessage(metadata.GetDisplayName()),
ValidationType = "isgreater",
};
rule.ValidationParameters.Add("otherproperty", otherPropertyName);
yield return rule;
}
}
What you want to do is test that if the value of EndDate is less than the value of StartDate, then the model is invalid, i.e. that the IsValid() method will throw a ValidationException
// Test that if the end date is less than the start date its invalid
[TestMethod]
[ExpectedException(typeof(ValidationException))]
public void TestEndDateIsInvalidIfLessThanStartDate()
{
// Initialize a model with invalid values
DateValTest model = new DateValTest(){ StartDate = DateTime.Today, EndDate = DateTime.Today.AddDays(-1) };
ValidationContext context = new ValidationContext(model);
DateGreaterThanAttribute attribute = new DateGreaterThanAttribute("StartDate");
attribute.Validate(model.EndDate, context);
}
When you run the test, if will succeed. Conversely if you were to initialize the model using
DateValTest model = new DateValTest(){ StartDate = DateTime.Today, EndDate = DateTime.Today.AddDays(1) };
the test would fail because the model is valid.