Finding sale anomalies across companies by day - ml.net

I am trying to find sales anomalies across companies using DetectIidChangePoint
This is my data model
public class TransactionResult
{
[LoadColumn(0)]
public string CompanyName;
[LoadColumn(1)]
public Int32 Month;
[LoadColumn(2)]
public Int32 Day;
[LoadColumn(3)]
public Int32 Year;
[LoadColumn(4)]
public Single Amount;
}
Does month/days/years that don't have data need a zero value provided, or is there lack of existence okay?
Does it matter what order the input the data in?
What does the change history length value do below?
I am using the DetectIidChangePoint
var iidChangePointEstimator = mlContext.Transforms.DetectIidChangePoint(outputColumnName: nameof(TransactionResultPrediction.Prediction), inputColumnName: nameof(TransactionResult.Amount), confidence: 95, changeHistoryLength: docSize / 4);
Thank you for the help in advance.

Related

Acumatica - Action "Reverse and Apply Memo" Default Post Period to active financial period

Can a customization be created to set the post period to the current active financial period after pressing the "Reverse and Apply to Memo" action in "Invoices and Memos" screen?
We've noticed the newly created credit memo defaults to the post period of the invoice which could be incorrect if it's credited in the following financial period.
The solution defined below was developed within Acumatica 20.102.0015 and changes the date and post period for the created credit memo on "Reverse and Apply Memo" action to the default of a new document instead of the date from the reversed invoice.
namespace AARAMPostPeriod
{
public class AAARInvoiceEntryExtension : PXGraphExtension<ARInvoiceEntry>
{
public delegate IEnumerable ReverseDocumentAndApplyToReversalIfNeededDel(PXAdapter adapter, ReverseInvoiceArgs reverseArgs);
[PXOverride]
public virtual IEnumerable ReverseDocumentAndApplyToReversalIfNeeded(PXAdapter adapter, ReverseInvoiceArgs reverseArgs, ReverseDocumentAndApplyToReversalIfNeededDel del)
{
if(reverseArgs.ApplyToOriginalDocument) reverseArgs.DateOption = ReverseInvoiceArgs.CopyOption.SetDefault;
return del(adapter, reverseArgs);
}
}
}
Default value for reverseArgs.DateOption is typically
ReverseInvoiceArgs.CopyOption.SetOriginal
You are talking about the receivables side of things, but I did something similar on the payables side. It isn't exactly what you are asking for, but it is too big for a comment.
You may be able to get the general idea from this and apply to your scenario. The approach I took was to check the period when releasing.
protected virtual void _(Events.FieldUpdated<APInvoice.finPeriodID> e)
{
APInvoice row = (APInvoice)e.Row;
CheckPeriod(e.Cache, row);
}
#region Release override
public delegate IEnumerable ReleaseDelegate(PXAdapter adapter);
[PXOverride]
public virtual IEnumerable Release(PXAdapter adapter, ReleaseDelegate baseMethod)
{
CheckPeriod(Base.Caches[typeof(APInvoice)], Base.Document.Current);
return baseMethod(adapter);
}
#endregion
protected virtual void CheckPeriod(PXCache cache, APInvoice invoice)
{
if (invoice?.FinPeriodID == null) return;
string currentPeriod = GetCurrentPeriod(invoice.BranchID);
if (currentPeriod != invoice.FinPeriodID)
{
PXUIFieldAttribute.SetError<APInvoice.finPeriodID>(cache, invoice, "Invalid period");
}
}
public virtual string GetCurrentPeriod(int? branchID)
{
PXResultset<Branch> Results = PXSelectJoin<GL.Branch,
InnerJoin<FinPeriod, On<FinPeriod.organizationID, Equal<Branch.organizationID>>>,
Where<Branch.branchID, Equal<Required<Branch.branchID>>,
And<FinPeriod.startDate, LessEqual<Required<FinPeriod.startDate>>,
And<FinPeriod.endDate, Greater<Required<FinPeriod.endDate>>>>>> // End Date is the date AFTER the period ends
.SelectSingleBound(Base, null, branchID, Base.Accessinfo.BusinessDate, Base.Accessinfo.BusinessDate);
if (Results != null)
{
foreach (PXResult<GL.Branch, FinPeriod> result in Results)
{
FinPeriod period = result;
return period.FinPeriodID;
}
}
return null;
}
As you can see, I put an override on the Release to perform my validation which sets an error condition if the period is not current. The validation is performed by retrieving the current period of the current business date and comparing to the period on the APInvoice.
You could explore leveraging GetCurrentPeriod from the example and put into an override on FieldDefaulting if it helps with your goal.

#Validate not skipping invalid rows when used with CsvRoutines in UniVocity parser

I am using Univocity parser version 2.7.3. I have a CSV file that has 1 Million records and might grow in future. I am reading only a few specific columns from the file and below are my requirements:
DO NOT store the CSV contents into memory at any point
Ignore/skip bean creation if either of latitude or longitude columns
in CSV are null/blank
To meet these requirements, I tried implementing CsvRoutines so that the CSV data is not copied over to memory. I am using #Validate annotation on both "Latitude" and "Longitude" fields and have used error handler to not throw back any exception so that the record will be skipped on validation failure.
Sample CSV:
#version:1.0
#timestamp:2017-05-29T23:22:22.320Z
#brand:test report
network_name,location_name,location_category,location_address,location_zipcode,location_phone_number,location_latitude,location_longitude,location_city,location_state_name,location_state_abbreviation,location_country,location_country_code,pricing_type,wep_key
"1 Free WiFi","Test Restaurant","Cafe / Restaurant","Marktplatz 18","1233","+41 263 34 05","1212.15","7.51","Basel","test","BE","India","DE","premium",""
"2 Free WiFi","Test Restaurant","Cafe / Restaurant","Zufikerstrasse 1","1111","+41 631 60 00","11.354","8.12","Bremgarten","test","AG","China","CH","premium",""
"3 Free WiFi","Test Restaurant","Cafe / Restaurant","Chemin de la Fontaine 10","1260","+41 22 361 69","12.34","11.23","Nyon","Vaud","VD","Switzerland","CH","premium",""
"!.oist*~","HoistGroup Office","Office","Chemin de I Etang","CH-1211","","","","test","test","GE","Switzerland","CH","premium",""
"test","tess's Takashiro","Cafe / Restaurant","Test 1-10","870-01","097-55-1808","","","Oita","Oita","OITA","Japan","JP","premium","1234B"
TestDTO.java
#Data
#NoArgsConstructor
#AllArgsConstructor
#JsonIgnoreProperties(ignoreUnknown = true)
public class TestDTO implements Serializable {
#Parsed(field = "location_name")
private String name;
#Parsed(field = "location_address")
private String addressLine1;
#Parsed(field = "location_city")
private String city;
#Parsed(field = "location_state_abbreviation")
private String state;
#Parsed(field = "location_country_code")
private String country;
#Parsed(field = "location_zipcode")
private String postalCode;
#Parsed(field = "location_latitude")
#Validate
private Double latitude;
#Parsed(field = "location_longitude")
#Validate
private Double longitude;
#Parsed(field = "network_name")
private String ssid;
}
Main.java
CsvParserSettings parserSettings = new CsvParserSettings();
parserSettings.detectFormatAutomatically();
parserSettings.setLineSeparatorDetectionEnabled(true);
parserSettings.setHeaderExtractionEnabled(true);
parserSettings.setSkipEmptyLines(true);
parserSettings.selectFields("network_name", "location_name","location_address", "location_zipcode",
"location_latitude", "location_longitude", "location_city","location_state_abbreviation", "location_country_code");
parserSettings.setProcessorErrorHandler(new RowProcessorErrorHandler() {
#Override
public void handleError(DataProcessingException error, Object[] inputRow, ParsingContext context) {
//do nothing
}
});
CsvRoutines parser = new CsvRoutines(parserSettings);
ResultIterator<TestDTO, ParsingContext> iterator = parser.iterate(TestDTO.class, new FileReader("c:\\users\\...\\test.csv")).iterator();
int i=0;
while(iterator.hasNext()) {
TestDTO dto = iterator.next();
if(dto.getLongitude() == null || dto.getLatitude() == null)
i++;
}
System.out.println("count=="+i);
Problem:
I actually expected the count to be zero since I have added error handler and haven't thrown back the data validation exception but seems thats not the case. I thought #Validate will throw back an exception when it encounters a record with either Latitude or Longitude as null (both the columns may be null in same record as well) which is handled and ignored/skipped at error handler.
Basically I do not want UniVocity to create and map unnecessary DTO objects in heap (and lead to out of memory) since there are chances that the incoming CSV file might have more than 200 or 300k records with either Latitude/Longitude as null.
I even tried adding custom validator in #Validate as well but in vain.
Could someone please let me know what am I missing here?
Author of the library here. You are doing everything right. This is a bug and I just opened this issue here to be resolved today.
The bug appears when you select fields: the reordering of values makes the validation run against something else (in my test, it validated the city instead of latitude).
In your case, just add the following line of code and it will work fine:
parserSettings.setColumnReorderingEnabled(false);
This will make the rows be generated with nulls where fields were not selected, instead of removing the nulls and reordering the values in the parsed row. It will avoid the bug and also make your program run slightly faster.
You will also need to test for null in the iteration bit:
TestDTO dto = iterator.next();
if(dto != null) { // dto may come null here due to validation
if (dto.longitude == null || dto.latitude == null)
i++;
}
}
Hope this helps and thank you for using our parsers!

Mysema QueryDSL JPAQuery for SQL groupby clause and order by

I have Items class:
#Entity
#Table(name ="Items")
Class Items{
#ID
private long id;
private String upc;
private long itemNo;
private int qty;
-----
}
I need to make below sql statement from JPAQuery of QueryDSL.
select itemNo, upc, count(*) t from Items group by ITEM_NO, UPC order by t;
QueryDSL sample needs modification for order by clause:
QItems items = QItems.items;
query.from(items)
.groupBy(items.itemNo,items.upc)
.orderby(<Dont Know How to sort on count>).list(items.itemNo,items.upc,items.count());
Need help to draft this query properly?
This should work
NumberPath<Long> count = Expressions.numberPath(Long.class, "c");
QItems items = QItems.items;
query.from(items)
.groupBy(items.itemNo,items.upc)
.orderby(count.asc())
.list(items.itemNo,items.upc,items.count().as(count));

JPA2 Criteria Query JOIN

I have following class ( entity )
#Entity
public class Magazine {
private int id ;
private String magazine;
//getters and setters
}
// tracking the magazine arrival
#Entity
public class MagazineIn {
private int id;
private java.util.Date inDate;
private java.util.Date magdate;
#OneToOne
private Magazine mag ;
//getters and setters
}
Now i want to get all arrival status of all magazine , whether
magazine has in or not
using criteria query
following is code
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Object[]> cq = cb.createQuery(Object[].class);
Root<MagazineIn> magIn = cq.from(MagazineIn.class);
Join<Magazine,MagazineIn> mag = magIn.join("mag" , JoinType.LEFT);
cq.multiselect(mag.get("magazine") , magIn.get("magdate") ,
magIn.get("inDate"));
TypedQuery<Object[]> q = em.createQuery(cq);
But i am not getting all the magazine listing with indate and magdate null.RIGHT JOIN is not supported. What's wrong?
If the entity relationships changes are not possible you could consider changing the root of your query to Magazine and use a subquery which correlates on the 'mag' variable.
What's wrong is that you would indeed need a right join to do this. You could just have the reverse OneToOne association from Magazine to MagazineIn, and query from the magazine to the magazineIn with a left join.
I have solved this using collection in Magazine
#OneToMany(mappedBy="mag",FETCHTYPE.LAZY)
List<MagazineIn> arrivals

EclipseLink/JPA2 ManyToOne bidirectional cascade persist issue

This is similar to How to cascade persist using JPA/EclipseLink
I have to entities like so. One is RoomEntity that has a one to many bi-directional relationship with ComputerEntity. eg. each room has 0..n computers.
#Entity
public class ComputerEntity implements Serializable {
#Id
#GeneratedValue(generator="computerSeq",strategy= GenerationType.SEQUENCE)
#SequenceGenerator(name="computerSeq",sequenceName="SEQUENCECOMPUTERID",allocationSize=1)
private long computerID;
#Column(name = "COMPUTERID")
public long getComputerID() {
return computerID;
}
public void setComputerID(long computerID) {
this.computerID = computerID;
}
private RoomEntity room;
#ManyToOne()
#JoinColumn(name = "ROOMID", referencedColumnName = "ROOMID")
public RoomEntity getRoom() {
return room;
}
public void setRoom(RoomEntity room) {
this.room = room;
}
} //ComputerEntity
#Entity
public class RoomEntity {
#Id
#GeneratedValue(generator="roomSeq",strategy= GenerationType.SEQUENCE)
#SequenceGenerator(name="roomSeq",sequenceName="SEQUENCEROOMID",allocationSize=1)
private long roomID;
#OneToMany(mappedBy = "room", cascade = {CascadeType.PERSIST, CascadeType.REMOVE})
private Set<ComputerEntity> computers;
#javax.persistence.Column(name = "ROOMID")
public long getRoomID() {
return roomID;
}
public void setRoomID(long roomID) {
this.roomID = roomID;
}
public Set<ComputerEntity> getComputers() {
return computers;
}
public void setComputers(Set<ComputerEntity> computers) {
for(ComputerEntity computer : computers) {
computer.setRoom(this);
}
this.computers = computers;
}
}//RoomEntity
When I try to persist an new room with a computer like so:
RoomEntity room = new RoomEntity();
room.setAdministrator("Fox Moulder");
room.setLocation("Area 51");
ComputerEntity computer1 = new ComputerEntity();
computer1.setDescription("Alienware area51 laptop");
Set<ComputerEntity> computers = new HashSet<ComputerEntity>();
computers.add(computer1);
room.setComputers(computers);
roomBean.createRoom(room);
roomBean is a Stateless EJB and roomBean.createRoom simply calls entityManager.persist(room). Since I have a CascadeType.PERSIST on the computers field of RoomEntity the ComptuerEntity is created. However, if I look at the room field of that ComputerEntity I see that room field is null. I would have assumed that Eclipselink would have automatically filled in the room since I have a bi-directional relationship. In order to have the room set in this way I had to add
for(ComputerEntity computer : computers) {
computer.setRoom(this);
}
to room.setComputers(...). Is this the correct approach or is there a way to get Eclipselink to automatically set this?
Thanks.
-Noah
There is no relationship maintenance in JPA. EJB 2.1 has shown that generic relationship management is for too inefficient and cumbersome.
I recommend adding an addComputer() method that would set the RoomEntity on the newly added ComputerEntity