Context Navigation

← Previous Ticket
Next Ticket →

#1820 closed defect (invalid)

text/x-gwt-rpc post failing sometimes

Reported by:	simon.verzijl.verzity.nl@…	Owned by:
Priority:	major	Milestone:
Component:	other	Version:	1.17.x
Keywords:		Cc:	simon.verzijl@…
uname -a:	Linux vm00004085 2.6.32-754.15.3.el6.x86_64 #1 SMP Thu Jun 13 22:49:44 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
nginx -V:	nginx version: nginx/1.17.1 (CentOS) built by gcc 4.4.7 20120313 (Red Hat 4.4.7-23) (GCC) built with OpenSSL 1.0.1e-fips 11 Feb 2013 TLS SNI support enabled configure arguments: --prefix=/appl/tib/nginx --modules-path=/appl/tib/nginx/modules --with-http_ssl_module --add-module=/tmp/ngx_devel_kit-0.3.1rc1 --add-module=/tmp/set-misc-nginx-module-0.32 --build=CentOS --with-select_module --with-poll_module --with-threads --with-file-aio --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-perl_modules_path=/usr/lib64/perl5 --with-perl=/usr/bin/perl --with-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_ssl_module --with-stream_realip_module --with-stream_geoip_module=dynamic --with-stream_ssl_preread_module --with-compat --with-openssl-opt=no-nextprotoneg --with-debug --http-log-path=/appl/tib/nginx/logs/access.log --error-log-path=/appl/tib/nginx/logs/error.log

Description

We have a Nginx reverse proxy configured in front of a TIBCO AMX-BPM backend system. This works fine for 99% but when repeating the same browser activity/same payload every now and then we get a HTTP/400 returned from Nginx.

In the access.log we see the following 'garbage' appearing :

10.29.140.34 - - [25/Jul/2019:07:50:15 +0200] "GET / HTTP/1.1" 200 396 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36"
10.29.140.34 - - [25/Jul/2019:08:25:09 +0200] "NameList\x22:[], \x22noTenantsObjNameList\x22:[]}}, {\x22$param\x22:\x22objectSearchFlag\x22, \x22mode\x22:\x22INOUT\x22, \x22type\x22:\x22BOOLEAN\x22, \x22$value\x22:false}, {\x22$param\x22:\x22CurrentObject\x22, \x22mode\x22:\x22INOUT\x22, \x22$value\x22:{\x22$type\x22:\x22nl.abnamro.dvb.objects.ObjectList\x22, \x22buildingUnits\x22:[], \x22pandUnits\x22:[], \x22huurders\x22:[]}}, {\x22$param\x22:\x22BrowseWithObjSearch\x22, \x22mode\x22:\x22INOUT\x22, \x22type\x22:\x22BOOLEAN\x22, \x22$value\x22:false}, {\x22$param\x22:\x22TotalTenantCount\x22, \x22mode\x22:\x22INOUT\x22, \x22type\x22:\x22INTEGER\x22, \x22$value\x22:\x2230\x22}]}|http://vm00004085.nl.eu.abnamro.com:8090/bpmresources/1.0.0.20190723113809420/openspaceGWTPull_DefaultChannel/DVB_NewAnalysisbs/OrchestrationProcess/SituationNew/SituationNew.gwt.json|java.util.ArrayList/4159755760|pvm:0a10a0|com.tibco.bpm.web.client.model.types.BusinessServiceTemplate/1666351301|nl.abnamro.dvb.launchDashboard/DVB_Dashboard|/nl.abnamro.dvb.launchDashboard/Process Packages/DVB_Dashboard.xpdl|MainDashboard|1.0.0.20190613125639319|openspaceGWTPull_DefaultChannel|1|2|3|4|3|5|6|7|5|8|9|10|0|A|11|0|11|0|0|0|11|0|0|12|A|0|13|14|15|16|17|18|" 400 157 "-" "-"

The actual payload from (Chrome) browser was this :

Request URL: http://vm00004085.nl.eu.abnamro.com:8090/bpm/BusinessServiceGadgetService
Request Method: POST
Status Code: 400 Bad Request
Remote Address: 10.240.48.246:8090
Referrer Policy: no-referrer-when-downgrade
Connection: close
Content-Length: 157
Content-Type: text/html
Date: Thu, 25 Jul 2019 06:25:09 GMT
Server: nginx/1.17.1
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Content-Length: 17380
Content-Type: text/x-gwt-rpc; charset=UTF-8
Cookie: JSESSIONID=90v78927nhxlvmqidekbfg8j; s_fid=7908B30519ED0DEB-3144CC6E1F994BDF; s_cahi=%5B%5B%27Direct%27%2C%271551088986099%27%5D%5D; _ga=GA1.2.1729516751.1558944295; s_new_repeat=1563429121440-Repeat
Host: vm00004085.nl.eu.abnamro.com:8090
Origin: http://vm00004085.nl.eu.abnamro.com:8090
Referer: http://vm00004085.nl.eu.abnamro.com:8090/openspace/?locale=en_US&ldap=true
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
X-GWT-Module-Base: http://vm00004085.nl.eu.abnamro.com:8090/openspace/
X-GWT-Permutation: 5F7B9556913825225907FA19BA273057
7|0|18|http://vm00004085.nl.eu.abnamro.com:8090/openspace/|35F26962680C2022276D5EE3DAB53951|com.tibco.openspace.services.shared.BusinessServiceGadgetService|updateBusinessService|com.tibco.bpm.web.client.model.types.PageData/2619873841|com.tibco.bpm.web.client.model.types.IBusinessServiceTemplate|java.lang.String/2004016611|pvm:001ga2.cm|{ "items":[{"$param":"Delete", "mode":"INOUT", "type":"BOOLEAN", "$value":false}, {"$param":"ButtonAction", "mode":"INOUT", "type":"STRING", "$value":"EditObject"}, {"$param":"ErrorMessage", "mode":"INOUT", "type":"STRING", "$value":""}, {"$param":"ActiveUser", "mode":"INOUT", "type":"STRING", "$value":"admin"}, {"$param":"ReportingServerDetails", "mode":"INOUT", "$value":[{"envId":"ST", "SSL":true, "host":"vm400016420.nl.eu.abnamro.com", "port":"9443", "baseURL":"/spotfire/wp/OpenAnalysis?file", "analysisName":"/ST/DVB1/Visualizations/", "$type":"nl.abnamro.dvb.gcdenums.ReportingServerDetails"}]}, {"$param":"SelectedObjectIndex", "mode":"INOUT", "type":"INTEGER", "$value":"13"}, {"$param":"Objects", "mode":"INOUT", "$value":[{"index":"13", "objectnaam":"Argonautenstraat 12 III", "plaats":"Amsterdam", "assettype":"Residentieel", "VVOm2":"58.0", "verhuurd":"58.0", "leeg":"0.0", "HIEurjaar":"4473.0", "HWEURjaar":"17400.0", "marktwaarde":"149761.0", "BAR":"0.11618512", "objectStatus":"Bestaand", "GCID":"501036", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0.7067245", "laatstedatumtaxatie":"2017-01-23", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Argonautenstraat 12 3", "VBOId":"0363010000572482", "VVO":"58", "energyClassId":"G", "weging":"7", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012083498", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630193", "huurderIndex":"0", "objectnaam":"Argonautenstraat 12 III", "adres":"12 3", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"4473.0", "HWEURjaar":"17400.0", "VVOm2":"58.0", "huurderStatus":"Bestaand", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"12", "objectnaam":"Argonautenstraat 14 I", "plaats":"Amsterdam", "assettype":"Residentieel", "VVOm2":"58.0", "verhuurd":"58.0", "leeg":"0.0", "HIEurjaar":"4817.0", "HWEURjaar":"17400.0", "marktwaarde":"161305.0", "BAR":"0.107870184", "objectStatus":"Bestaand", "GCID":"501036", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0.7612008", "laatstedatumtaxatie":"2017-01-23", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Argonautenstraat 14 1", "VBOId":"0363010000572485", "VVO":"58", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012118353", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630186", "huurderIndex":"0", "objectnaam":"Argonautenstraat 14 I", "adres":"14 1", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"4817.0", "HWEURjaar":"17400.0", "VVOm2":"58.0", "huurderStatus":"Bestaand", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"15", "objectnaam":"Argonautenstraat 8 HS", "plaats":"Amsterdam", "assettype":"Residentieel", "VVOm2":"58.0", "verhuurd":"58.0", "leeg":"0.0", "HIEurjaar":"5418.0", "HWEURjaar":"18850.0", "marktwaarde":"181416.0", "BAR":"0.103904836", "objectStatus":"Bestaand", "GCID":"501036", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0.85610497", "laatstedatumtaxatie":"2017-01-23", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Argonautenstraat 8 H", "VBOId":"0363010000572469", "VVO":"58", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012139071", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630207", "huurderIndex":"0", "objectnaam":"Argonautenstraat 8 HS", "adres":"8 H", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"5418.0", "HWEURjaar":"18850.0", "VVOm2":"58.0", "huurderStatus":"Bestaand", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"14", "objectnaam":"Argonautenstraat 8 II", "plaats":"Amsterdam", "assettype":"Residentieel", "VVOm2":"58.0", "verhuurd":"58.0", "leeg":"0.0", "HIEurjaar":"8884.0", "HWEURjaar":"17400.0", "marktwaarde":"181732.0", "BAR":"0.095745385", "objectStatus":"Bestaand", "GCID":"501036", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0.8575962", "laatstedatumtaxatie":"2017-01-23", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Argonautenstraat 8 2", "VBOId":"0363010000572471", "VVO":"58", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012139071", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630200", "huurderIndex":"0", "objectnaam":"Argonautenstraat 8 II", "adres":"8 2", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"8884.0", "HWEURjaar":"17400.0", "VVOm2":"58.0", "huurderStatus":"Bestaand", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"11", "objectnaam":"Coppelstockstraat 43 HS", "plaats":"Amsterdam", "assettype":"Residentieel", "VVOm2":"64.0", "verhuurd":"64.0", "leeg":"0.0", "HIEurjaar":"4263.0", "HWEURjaar":"17600.0", "marktwaarde":"148131.0", "BAR":"0.11881375", "objectStatus":"Bestaand", "GCID":"316920", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0.69903255", "laatstedatumtaxatie":"2017-01-23", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Coppelstockstraat 43 H", "VBOId":"0363010000608718", "VVO":"51", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012121115", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630179", "huurderIndex":"0", "objectnaam":"Coppelstockstraat 43 HS", "adres":"43 H", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"4263.0", "HWEURjaar":"17600.0", "VVOm2":"64.0", "huurderStatus":"Bestaand", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"4", "objectnaam":"Groene Kadeweg 51 en 53", "plaats":"Almere", "assettype":"Overig", "VVOm2":"1195.0", "verhuurd":"1195.0", "leeg":"0.0", "HIEurjaar":"207950.0", "HWEURjaar":"179250.0", "marktwaarde":"2430000.0", "BAR":"0.073765434", "objectStatus":"Bestaand", "GCID":"501037", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Savills Taxaties", "weging":"11.467208", "laatstedatumtaxatie":"2016-03-14", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Groene Kadeweg 51", "VBOId":"0034010000085710", "VVO":"238.0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0034100000104752", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630126", "huurderIndex":"0", "objectnaam":"Groene Kadeweg 51 en 53", "adres":"51", "entiteitHuurder":"Stichting Dierenthuis", "huurinkomstenperjaar":"207950.0", "einddatumContract":"2037-12-31", "looptijd":"18.798084", "HWEURjaar":"179250.0", "VVOm2":"1195.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"10", "objectnaam":"Groenedaalstraat 44 II", "plaats":"Amsterdam", "assettype":"Residentieel", "VVOm2":"56.0", "verhuurd":"56.0", "leeg":"0.0", "HIEurjaar":"3684.0", "HWEURjaar":"16800.0", "marktwaarde":"150763.0", "BAR":"0.11143318", "objectStatus":"Bestaand", "GCID":"316921", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0.71145296", "laatstedatumtaxatie":"2017-01-23", "tenantCount":"1", "waarderingCount":"1", "waarderingStatus":"New", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Groenendaalstraat 44 1", "VBOId":"0363010000651783", "VVO":"52", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}, {"investmentObject":"Groenendaalstraat 44 2", "VBOId":"0363010000651784", "VVO":"52", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012071382", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630172", "huurderIndex":"0", "objectnaam":"Groenedaalstraat 44 II", "adres":"44 2", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"3684.0", "HWEURjaar":"16800.0", "VVOm2":"56.0", "huurderStatus":"Bestaand", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"19", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "plaats":"Haarlem", "assettype":"Winkel", "VVOm2":"813.0", "verhuurd":"813.0", "leeg":"0.0", "HIEurjaar":"183827.0", "HWEURjaar":"196236.0", "marktwaarde":"2870000.0", "BAR":"0.06837491", "objectStatus":"Bestaand", "GCID":"501038", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"13.543574", "laatstedatumtaxatie":"2017-02-03", "tenantCount":"6", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Grote Houtstraat 102 A", "VBOId":"0392010000022551", "VVO":"43", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}, {"investmentObject":"Grote Houtstraat 102 ARD", "VBOId":"0392010000022552", "VVO":"111", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}, {"investmentObject":"Grote Houtstraat 104", "VBOId":"0392010000022555", "VVO":"375", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0392100000038683", "$type":"nl.abnamro.dvb.objects.PandUnit"}, {"pandID":"0392100000036113", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630235", "huurderIndex":"1", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "adres":"104", "entiteitHuurder":"Dhr. Moshe Beaucaire  dhr. F.P.R.M. Lem", "huurinkomstenperjaar":"15621.0", "einddatumContract":"2019-07-31", "looptijd":"0.3778234", "HWEURjaar":"16225.0", "VVOm2":"59.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}, {"objectIndex":"630235", "huurderIndex":"2", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "adres":"104", "entiteitHuurder":"Holland  Barrett", "huurinkomstenperjaar":"99992.0", "einddatumContract":"2019-07-31", "looptijd":"0.3778234", "HWEURjaar":"87346.0", "VVOm2":"367.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}, {"objectIndex":"630235", "huurderIndex":"3", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "adres":"102 ARD", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"9194.0", "HWEURjaar":"21875.0", "VVOm2":"175.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}, {"objectIndex":"630235", "huurderIndex":"4", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "adres":"102 ARD", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"7742.0", "HWEURjaar":"15540.0", "VVOm2":"111.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}, {"objectIndex":"630235", "huurderIndex":"5", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "adres":"102 A", "entiteitHuurder":"Samag Haarlem B.V. i.o.", "huurinkomstenperjaar":"31299.0", "einddatumContract":"2019-03-31", "looptijd":"0.043805614", "HWEURjaar":"35100.0", "VVOm2":"39.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}, {"objectIndex":"630235", "huurderIndex":"0", "objectnaam":"Grote Houtstraat 104  - 102 ARD", "adres":"104", "entiteitHuurder":"Jac. Woestenburg Haarlem B.V.", "huurinkomstenperjaar":"19979.0", "einddatumContract":"2019-12-31", "looptijd":"0.7967146", "HWEURjaar":"20150.0", "VVOm2":"62.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"3", "objectnaam":"Grote Houtstraat 146- 146A", "plaats":"Haarlem", "assettype":"Horeca", "VVOm2":"104.0", "verhuurd":"104.0", "leeg":"0.0", "HIEurjaar":"30100.0", "HWEURjaar":"39325.0", "marktwaarde":"0.0", "BAR":"0.0", "objectStatus":"Bestaand", "GCID":"501034", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Cushman & Wakefield V.O.F.", "weging":"0", "laatstedatumtaxatie":"2017-02-03", "tenantCount":"2", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Grote Houtstraat 146", "VBOId":"0392010000022654", "VVO":"36.0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}, {"investmentObject":"Grote Houtstraat 146 A", "VBOId":"0392010000083494", "VVO":"65.0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0392100000035425", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630119", "huurderIndex":"1", "objectnaam":"Grote Houtstraat 146- 146A", "adres":"146", "entiteitHuurder":"Dhr. V.C.M. Bartels", "huurinkomstenperjaar":"25467.0", "einddatumContract":"2019-10-31", "looptijd":"0.62970567", "HWEURjaar":"26325.0", "VVOm2":"39.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}, {"objectIndex":"630119", "huurderIndex":"0", "objectnaam":"Grote Houtstraat 146- 146A", "adres":"146 A", "entiteitHuurder":"Particulier", "huurinkomstenperjaar":"4633.0", "HWEURjaar":"13000.0", "VVOm2":"65.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}]}, {"index":"20", "objectnaam":"Korte Leidsedwarsstraat 18-1", "plaats":"Amsterdam", "assettype":"Winkel", "VVOm2":"525.0", "verhuurd":"525.0", "leeg":"0.0", "HIEurjaar":"161000.0", "HWEURjaar":"186900.0", "marktwaarde":"3825000.0", "BAR":"0.048862744", "objectStatus":"Bestaand", "GCID":"501040", "doel":"Belegging", "exploitatielasten":"15.0", "typewaardering":"Jones Lang LaSalle", "weging":"18.050236", "laatstedatumtaxatie":"2018-01-11", "tenantCount":"1", "waarderingCount":"1", "$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[{"investmentObject":"Korte Leidsedwarsstraat 18", "VBOId":"0363010000713998", "VVO":"2763", "energyClassId":"Onbekend", "weging":"0", "$type":"nl.abnamro.dvb.objects.BuildingUnit"}], "pandUnits":[{"pandID":"0363100012168737", "$type":"nl.abnamro.dvb.objects.PandUnit"}], "huurders":[{"objectIndex":"630081", "huurderIndex":"0", "objectnaam":"Korte Leidsedwarsstraat 18-1", "adres":"18", "entiteitHuurder":"Jimmy Woo BV", "huurinkomstenperjaar":"161000.0", "einddatumContract":"2019-07-31", "looptijd":"0.37770933", "HWEURjaar":"186900.0", "VVOm2":"525.0", "huurderStatus":"Gewijzigd", "$type":"nl.abnamro.dvb.objects.Huurder"}]}]}, {"$param":"SelectedHuurderIndex", "mode":"INOUT", "type":"INTEGER", "$value":"-1"}, {"$param":"FacilitiesList", "mode":"INOUT", "$value":[{"facilityIndex":"0", "nieuwBestaand":"Bestaand", "kredietlimiet":"1.0", "renteType":"Vast", "funding":"0.1", "nettoMarge":"0.1", "FTPLiquidity":"0.1", "rentetotaal":"0.30000000000000004", "renteBedrag":"0.0030000000000000005", "percentage":"100.0", "bedrag":"1.0", "expiratiedatum":"2024-04-01", "startdatum":"2019-04-01", "$type":"nl.abnamro.dvb.facilitylist.FacilityList"}]}, {"$param":"SelectedFacilityIndex", "mode":"INOUT", "type":"INTEGER", "$value":"-1"}, {"$param":"ProcessState", "mode":"INOUT", "type":"STRING", "$value":"INITIALANALYSISMADE"}, {"$param":"AanvraagDoel", "mode":"INOUT", "type":"STRING", "$value":"Revisie"}, {"$param":"KlantNaam", "mode":"INOUT", "type":"STRING", "$value":"Nwyany Q.C."}, {"$param":"StartObjectIndex", "mode":"INOUT", "type":"INTEGER", "$value":"0"}, {"$param":"HasMoreObjects", "mode":"INOUT", "type":"BOOLEAN", "$value":true}, {"$param":"TotalObjectCount", "mode":"INOUT", "type":"INTEGER", "$value":"22"}, {"$param":"PageSize", "mode":"INOUT", "type":"INTEGER", "$value":"10"}, {"$param":"CalculatedValues", "mode":"INOUT", "$value":{"endContractBreachedTenant":"3", "objectStatusPerObjects":["Bestaand"], "totalMaarketwaarde":"2.119086E7", "totalHI":"1150086.0", "totalHW":"1363225.0", "totalVVO":"5168.0", "totalLeeg":"0.0", "totalObjectCount":"22", "totalDummyCount":"0", "$type":"nl.abnamro.dvb.paginatedlist.VastgoedCalculatedData", "noAppraisalObjNameList":[], "noTenantsObjNameList":[]}}, {"$param":"objectSearchFlag", "mode":"INOUT", "type":"BOOLEAN", "$value":false}, {"$param":"CurrentObject", "mode":"INOUT", "$value":{"$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[], "pandUnits":[], "huurders":[]}}, {"$param":"BrowseWithObjSearch", "mode":"INOUT", "type":"BOOLEAN", "$value":false}, {"$param":"TotalTenantCount", "mode":"INOUT", "type":"INTEGER", "$value":"30"}]}|http://vm00004085.nl.eu.abnamro.com:8090/bpmresources/1.0.0.20190723113809420/openspaceGWTPull_DefaultChannel/DVB_NewAnalysisbs/OrchestrationProcess/SituationNew/SituationNew.gwt.json|java.util.ArrayList/4159755760|pvm:0a10a0|com.tibco.bpm.web.client.model.types.BusinessServiceTemplate/1666351301|nl.abnamro.dvb.launchDashboard/DVB_Dashboard|/nl.abnamro.dvb.launchDashboard/Process Packages/DVB_Dashboard.xpdl|MainDashboard|1.0.0.20190613125639319|openspaceGWTPull_DefaultChannel|1|2|3|4|3|5|6|7|5|8|9|10|0|A|11|0|11|0|0|0|11|0|0|12|A|0|13|14|15|16|17|18|

nginx.conf:

#user  nobody;
worker_processes  1;

error_log  logs/error.log info;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
#    keepalive_timeout 65;

    sendfile        on;
    gzip  on;

    # http://nginx.org/en/docs/varindex.html for all vars
    log_format custom '$remote_addr $http_host $remote_user $timestr "$request" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" $request_time $request_length';

    server {
      listen 10.240.48.246:8090;
      if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})") {
        set $year $1;
        set $month $2;
        set $day $3;
      }
      chunked_transfer_encoding off;
      client_max_body_size    10M;
      client_body_buffer_size 8k;
      client_body_in_file_only on;

      set_formatted_local_time $timestr "%Y-%m-%d %H:%M:%S";
      location / {
        proxy_pass http://127.0.0.1:8090;
        proxy_set_header Host $host:$server_port;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        access_log logs/access_log_DVB.$year.$month.$day custom;
      }
    }


}

Some other times the error is not 'garbage' in the access.log, but the POST body itself is incomplete, or rather wrapped arround/shifted in client_body_temp/.

I have not been able to reproduce this error, but when I do I will add it as well.

Attachments (1)

error_capture.pcap (100.5 KB ) - added by simon.verzijl.verzity.nl@… 5 years ago.

Download all attachments as: .zip

Change History (4)

comment:1 by Maxim Dounin, 5 years ago

Please provide debuging log for a connection with the error. Additionally, tcpdump of the connection might be helpful as well.

by simon.verzijl.verzity.nl@…, 5 years ago

Attachment:	error_capture.pcap added

comment:2 by simon.verzijl.verzity.nl@…, 5 years ago

From the debug log :

2019/07/29 08:23:28 [debug] 32748#32748: epoll: fd:6 ev:0001 d:00007F9683164010
2019/07/29 08:23:28 [debug] 32748#32748: accept on 10.240.48.246:8090, ready: 0
2019/07/29 08:23:28 [debug] 32748#32748: posix_memalign: 000000000236D5B0:512 @16
2019/07/29 08:23:28 [debug] 32748#32748: *6235 accept: 10.29.140.34:52399 fd:3
2019/07/29 08:23:28 [debug] 32748#32748: *6235 event timer add: 3: 60000:1918101906
2019/07/29 08:23:28 [debug] 32748#32748: *6235 reusable connection: 1
2019/07/29 08:23:28 [debug] 32748#32748: *6235 epoll add event: fd:3 op:1 ev:80002001
2019/07/29 08:23:28 [debug] 32748#32748: timer delta: 3142
2019/07/29 08:23:28 [debug] 32748#32748: worker cycle
2019/07/29 08:23:28 [debug] 32748#32748: epoll timer: 60000
2019/07/29 08:23:28 [debug] 32748#32748: epoll: fd:3 ev:0001 d:00007F9683164200
2019/07/29 08:23:28 [debug] 32748#32748: *6235 http wait request handler
2019/07/29 08:23:28 [debug] 32748#32748: *6235 malloc: 00000000023ABAC0:1024
2019/07/29 08:23:28 [debug] 32748#32748: *6235 recv: eof:0, avail:1
2019/07/29 08:23:28 [debug] 32748#32748: *6235 recv: fd:3 994 of 1024
2019/07/29 08:23:28 [debug] 32748#32748: *6235 reusable connection: 0
2019/07/29 08:23:28 [debug] 32748#32748: *6235 posix_memalign: 000000000239F730:4096 @16
2019/07/29 08:23:28 [debug] 32748#32748: *6235 http process request line
2019/07/29 08:23:28 [info] 32748#32748: *6235 client sent invalid method while reading client request line, client: 10.29.140.34, server: , request: "ameList":[], "noTenantsObjNameList":[]}}, {"$param":"objectSearchFlag", "mode":"INOUT", "type":"BOOLEAN", "$value":false}, {"$param":"CurrentObject", "mode":"INOUT", "$value"
:{"$type":"nl.abnamro.dvb.objects.ObjectList", "buildingUnits":[], "pandUnits":[], "huurders":[]}}, {"$param":"BrowseWithObjSearch", "mode":"INOUT", "type":"BOOLEAN", "$value":false}, {"$param":"TotalTenantCount", "mode":"INOUT", "type":"INTEGER", "$value":"30"}]}|http://vm00004085.nl.eu.abnamro.com:8090/bpmresources/1.0.0
.20190726114932914/openspaceGWTPull_DefaultChannel/DVB_NewAnalysisbs/OrchestrationProcess/SituationNew/SituationNew.gwt.json|java.util.ArrayList/4159755760|pvm:0a102|com.tibco.bpm.web.client.model.types.BusinessServiceTemplate/1666351301|nl.abnamro.dvb.launchDashboard/DVB_Dashboard|/nl.abnamro.dvb.launchDashboard/Process P
ackages/DVB_Dashboard.xpdl|MainDashboard|1.0.0.20190613125639319|openspaceGWTPull_DefaultChannel|1|2|3|4|3|5|6|7|5|8|9|10|0|A|11|0|11|0|0|0|11|0|0|12|A|0|13|14|15|16|17|18|"
2019/07/29 08:23:28 [debug] 32748#32748: *6235 http finalize request: 400, "?" a:1, c:1
2019/07/29 08:23:28 [debug] 32748#32748: *6235 event timer del: 3: 1918101906
2019/07/29 08:23:28 [debug] 32748#32748: *6235 http special response: 400, "?"
2019/07/29 08:23:28 [debug] 32748#32748: *6235 http set discard body
2019/07/29 08:23:28 [debug] 32748#32748: *6235 HTTP/1.1 400 Bad Request^M
Server: nginx/1.17.1^M
Date: Mon, 29 Jul 2019 06:23:28 GMT^M
Content-Type: text/html^M
Content-Length: 157^M
Connection: close^M

I've also attached a dump using Wireshark in PCAP format.
At 08:23:18,770070 (frame 6) it shows a succesfull POST (14 frames in total) with all packets in the right order.

At 08:23.26,277152 (frame 36)the unsuccesfull POST starts (14 frames in total). This should be the identical payload (only difference is some id 'pvm:001g4.e6' vs 'pvm:001g4.dq').
But in this case the first frame that is shown in wireshark should actually be sent the last. The next frame (frame 37) is the actual start of the POST.

So it looks like the (Chrome) browser is just sending the packets in wrong order (?) in this case if I interpret this correctly.
In the first (good) POST there is also just 1 frame being sent with PSH,ACK. In the second (bad) POST the first 2 frames are sent with PSH,ACK. Not sure what this means though.

Can you confirm this cannot be solved on Nginx side but is a client issue ?

(PS: When we remove the reverse proxy and have the browser connect to the AMX-BPM backend directly, we do not experience any issues. I would have expected similar issues to occur then as well)

Last edited 5 years ago by simon.verzijl.verzity.nl@… (previous) (diff)

comment:3 by Maxim Dounin, 5 years ago

Resolution:	→ invalid
Status:	new → closed

Clearly at 08:23:26.272934 client opens a new connection and starts sending garbage in it:

08:23:26.272934 IP 10.29.140.34.52399 > 10.240.48.246.8090: Flags [S], seq 3487973653, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
        0x0000:  4500 0034 1c90 4000 8006 0c0f 0a1d 8c22  E..4..@........"
        0x0010:  0af0 30f6 ccaf 1f9a cfe6 4115 0000 0000  ..0.......A.....
        0x0020:  8002 faf0 a4b4 0000 0204 05b4 0103 0308  ................
        0x0030:  0101 0402                                ....
08:23:26.274978 IP 10.29.140.34.52399 > 10.240.48.246.8090: Flags [.], ack 2158641205, win 260, length 0
        0x0000:  4500 0028 1c91 4000 8006 0c1a 0a1d 8c22  E..(..@........"
        0x0010:  0af0 30f6 ccaf 1f9a cfe6 4116 80aa 4035  ..0.......A...@5
        0x0020:  5010 0104 1e85 0000                      P.......
08:23:26.277152 IP 10.29.140.34.52399 > 10.240.48.246.8090: Flags [P.], seq 0:994, ack 1, win 260, length 994
        0x0000:  4500 040a 1c92 4000 8006 0837 0a1d 8c22  E.....@....7..."
        0x0010:  0af0 30f6 ccaf 1f9a cfe6 4116 80aa 4035  ..0.......A...@5
        0x0020:  5018 0104 b26a 0000 616d 654c 6973 7422  P....j..ameList"
        0x0030:  3a5b 5d2c 2022 6e6f 5465 6e61 6e74 734f  :[],."noTenantsO
        0x0040:  626a 4e61 6d65 4c69 7374 223a 5b5d 7d7d  bjNameList":[]}}
        0x0050:  2c20 7b22 2470 6172 616d 223a 226f 626a  ,.{"$param":"obj
        0x0060:  6563 7453 6561 7263 6846 6c61 6722 2c20  ectSearchFlag",.
        0x0070:  226d 6f64 6522 3a22 494e 4f55 5422 2c20  "mode":"INOUT",.
        0x0080:  2274 7970 6522 3a22 424f 4f4c 4541 4e22  "type":"BOOLEAN"
        ...
08:23:26.277242 IP 10.29.140.34.52399 > 10.240.48.246.8090: Flags [P.], seq 994:2357, ack 1, win 260, length 1363
        0x0000:  4500 057b 1c93 4000 8006 06c5 0a1d 8c22  E..{..@........"
        0x0010:  0af0 30f6 ccaf 1f9a cfe6 44f8 80aa 4035  ..0.......D...@5
        0x0020:  5018 0104 21e0 0000 504f 5354 202f 6270  P...!...POST./bp
        0x0030:  6d2f 4275 7369 6e65 7373 5365 7276 6963  m/BusinessServic
        0x0040:  6547 6164 6765 7453 6572 7669 6365 2048  eGadgetService.H
        0x0050:  5454 502f 312e 310d 0a48 6f73 743a 2076  TTP/1.1..Host:.v
        0x0060:  6d30 3030 3034 3038 352e 6e6c 2e65 752e  m00004085.nl.eu.
        0x0070:  6162 6e61 6d72 6f2e 636f 6d3a 3830 3930  abnamro.com:8090
        ...

No idea why this happens, and why it doesn't happen to you with direct connection - there may be a lot of factors, including timing ones. Either way, this is a client bug and it needs to be fixed on the client side.

Note: See TracTickets for help on using tickets.

Download in other formats: