IDempiere/FullMeeting20141224

⇐ Table of Contents | Full Meeting Minutes | Full Meeting 2014-12-24

CarlosRuiz: Good Morning
TurkerTunali: good morning
TurkerTunali: is there anyone who user SmartPOS?
CarlosRuiz: well - smartjsp is the author
CarlosRuiz: I'm reviewing https://idempiere.atlassian.net/browse/IDEMPIERE-644
smartjsp: Hi Turker ..
smartjsp: You can ask your smartpos questions within the sourceforge forums, http://sourceforge.net/p/smart-pos/discussion/?source=navbar
smartjsp: Good Morning everyone and Merry Christmas ...
smartjsp: Question about how to propagate Idempiere AD changes (such as: messages, sysconfigs, menus, references and so on) from one instance (dev for example) to other one (prod) ,,,is there any way to generate migration scripts (sql) for those changes ?.... currently we have them identified with an specific entity type ...and currently we perform those changes manually... any suggestions ?
smartjsp: our new SmartPOS 3.0 (OSGi plugin) requires a lot of AD changes (meesages, menus, references and so on) ... and in order to decouple the OSGi plugin as much as possible from an specific database, we want to include those AD changes , perhaps using apackin or sql scripts so the end users can get those new AD changes and SQL customizations within their Idempiere during the OSGi install ... then we are looking for a way to generate
Deepak: carlos, Sorry was away
Deepak: I reopend IDEMPIERE-644
Deepak: As the solution accepted was not good, We should have to come with better approach to manage active connections
Deepak: 2 hours is quite big if Trx are not handled properly in some code and 2 hours is quite small for long running processes
Deepak: smartjsp, Please note that iDempiere supports both migration scripts and 2pack, If your changes are going to be part of iDempiere core, it should be migrations script
Deepak: If you are writting module, you need to do 2pack and you can use iDempiere activator to auto apply 2packs on deploying plugins
ocurieles_DCS: HI boys
smartjsp: thanks Deepak .. in that case how can we generate the 2pack from the current non-standard AD objects ? do we have any way to generate the xml files from specific entity types ? we have seen packout option for menus, windows and tables for example, but not for messages, references for instance ? any suggestions ?
Deepak: smarjsp, One esiest way to 2pack any data is export button on windows
Deepak: It can export current window data as per filter condition in 2pack
Deepak: Carlos, ut?
smartjsp: perfect .. thanks Deepak...
CarlosRuiz: yes Deepak - making some changes to troubleshoot IDEMPIERE-644
CarlosRuiz: Deepak, I made some changes to verify the behavior - make a process infinitely insert into t_selection table - set the trx timeout to 2 min and set the monitor timeout to verify every minute
CarlosRuiz: results are very bad :-(
CarlosRuiz: the trx was closed by the TrxMonitor without any UI feedback
CarlosRuiz: the pending records were committed
CarlosRuiz: and now the process is in a loop writing in the log
CarlosRuiz: "Transaction closed or never opened"
CarlosRuiz: more weird - the process insert with DB.executeUpdateEx - but exception is not raised
Deepak: Carlos, Exactly that what I pointed
Deepak: While opening this issue
Deepak: It makes DB in consistant
Deepak: I am researching how we can make it to update m_startTime whenever connection is used, So we can set small time out like 10 minutes
Deepak: And it do not recover transactions untill it is in use
CarlosRuiz: agree with you about setting priority to Critical - if it can cause data corruption
CarlosRuiz: firstly we need to solve that - it must rollback - I'm checking with a new Trx.rollbackAndClose() ... looking for the results
Deepak: I am thinking that we should update M_start time on getConnection()
Deepak: So it is always latestly used
CarlosRuiz: also - we must check the behavior if the process is ran in background
Deepak: but my worry is some code use like conn = trx.getConnection(); like 2pack code
Deepak: carlos, Trx.rollbackAndClose() will not make DB inconsistent, but some of process never able to finish
Deepak: So that is another issue we may have
CarlosRuiz: yes - but that's a different issue - the Critical thing is the data inconsistency
CarlosRuiz: for the process not finishing there are workarounds
Deepak: Also Carlos, we need to throw exception or log sever level so user notice what is going wrong
CarlosRuiz: note also the same process can run in 1 minute in a tenant - and 2 hours in a different tenant - because of data volume - so setting a timeout per process maybe is not the best
Deepak: I am not agree with time out per process thought
Deepak: But those are Hengsin's view
Deepak: If we are not handling trx properly, it may cause trx pool to consumed withing 30 minutes, then no way to wait for 2 hours
Deepak: Only what is good parameter to consider expire is based on last access time
Deepak: And short timeout line 5-10 minutes
CarlosRuiz: also - wondering what happen when a db trx is running and the UI session timeout?
Deepak: One of dev in our team has that observation with Adempiere or compiere, but my observation with iDempiere says process keep running in background
Deepak: We are getting this issue after 2 hours when running import inventory while we have 1 hour session time out, that is enough to say that UI timeout is not causing process to stops
Deepak: Yes, but there will be exception at end of finishing of process when process dialog try to unlock
Deepak: Carlos, would you like this concept? despite fix you are making for roleback and close, Lets try to reset m_starttime whenever getConnection, commit, rollback methods on Trx getting called
Deepak: Changing timeout to be small like 10 minutes
CarlosRuiz: I'm not sure - maybe is not good - in such case you can have processes running for 100 hours
Deepak: Managing process is different then Trx
Deepak: In case of process, we may have to add process management to kill wrong running process in safe manner
CarlosRuiz: if you need such long running process - then you can apply the workaround to that specific process
CarlosRuiz: I think the default timeout must be restrictive - and you make it permissive with workarounds - not making the default permissive
CarlosRuiz: just found that closing the transaction raises a fireAfterCloseEvent - calling the TrxEventListener - but SvrProcess is not listening to that
CarlosRuiz: so - I implemented rollback - but it doesn't work well - as the rollback happens - and the process keeps running in "autocommit" mode
CarlosRuiz: BTW guys - I started logging IRC all day and publishing it here
CarlosRuiz: http://www.globalqss.com/idempiere/irclog/
CarlosRuiz: so :-) be careful with what you say - don't put passwords or sensitive information here
Deepak: Then all process need to change to listen to trx timeout and need to take need full action
CarlosRuiz: and if you write mail addresses here - then you could expect some spam
CarlosRuiz: yes Deepak - I think SvrProcess must do that
Deepak: Carlos, Are you going to commit that?
CarlosRuiz: no - testing here - I'll add a patch with comments
CarlosRuiz: I'm going out soon for christmas - so, at least I'll put a WIP patch
red1: Merry Christmas CarlosRuiz
CarlosRuiz: same for you red1 - and merry christmas to all you guys
red1: And everyone else
Deepak: Marry Christmas to all
norbertbede: same from slovakia :)
edilsondneto: Marry Christmas to all
CarlosRuiz: ready - patches posted in JIRA - thanks guys - merry christmas - I'm going out - c u then