Error from remote server

Quote

Guest
30 Mar 2016 14:35

Hi there,

I'm stil trying to improve my databse with search options (already had some help here last week and before). As I try to run the Cobalt reindex tool, I get this message:

The proxy server received an invalid response from an upstream server. The proxy server could not handle the request POST /administrator/index.php.

Reason: Error reading from remote server

Sergey already advised me to Google for this error, so I did, and I asked the helpdesk of my provider how to solve this problem (it seems I can't fix it myself, I know it has something to do with the 30 seconds that is the max that one can use on the server). The helpdesk is very slow, and after some answers that didn't help, they know advise me to ask the question here. So I hope someone here has a solution, because I can't make my search options work as they should if I can't use the reindex tool. Anybody? Please?

Manon

Last Modified: 04 Apr 2016

Quote

Sackgesicht VIP

Total posts: 1,636
30 Mar 2016 19:54

Manon,

Reindex has always been a problem when your section reaches a certain number of records. It depends on your PHP server settings, the way Cobalt reindexes its data, the Cobalt data structure itself and how much data (all fields together) every record stores.

For the PHP settings, the reindex script tries to override the max_execution_time setting, the bigger problem is the PHP allocated memory, based on the way the reindex works now.

The script reads ALL content of ALL records from a section together at once which can easily result in a memory problem, once you have a certain number of records in your section.

Using PHP 7 can help here a little, since it will allow reindexing more records based on lower memory consumption.

Another issue is the number of indices the #js_res_record table uses. Reindexing will be a massive process for the database here. Even simple inserts can slow down everything with several users adding records at the same time. If Cobalt development continues, improvement of the index handling should be a priority to allow Cobalt to be used with bigger datasets.

The best advice for your reindex problem would be an improved reindex script (your own script). To study the existing process, see /plugins/mint/toolset/toolset.php

Quote

Sackgesicht VIP

Total posts: 1,636
31 Mar 2016 10:39

I just remember that i had a similar error message when creating complex PDF lists from Cobalt on a local Mac Server. --> Proxy Error 502

I solved it in adding the ProxyTimeout based on the suggestions of this discussion: Click here to link...

Important point was, to restart the Apache instance which provides the proxy service. link

Quote

Sergey

Total posts: 13,748
31 Mar 2016 12:55

Can you adjust PHP settings? what if you create php.ini in the root of your site?

Sackgesicht If Cobalt development continues, improvement of the index handling should be a priority to allow Cobalt to be used with bigger datasets.

I remember that.

Quote

meijermanon
Total posts: 14
03 Apr 2016 08:17

Update: the helpdesk finally advised me to copy the whole database to a local webserver (XAMPP) and then run the reindextool. I'v tried this, and the good news is: the reindex tool works fine. It takes 2 minutes, and then I get the message '10.650 records reindexed' (or soemthing like that). So that's in my case not a problem. The rest of the problem is not solved yet, but for those who are interested, I thought I'd let you know.

Quote

Sackgesicht VIP

Total posts: 1,636
04 Apr 2016 00:59

Most of the time the reindex produces an "Out of Memory" error. I had to increase the memory limit to 1GB from the default 128MB to be able to reindex some sections.

Reindex improvement:

PHP7 helps here a lot. Another thing to reduce the memory consumption is to adjust the reindex SQL query on line 246. Removing the unnecessary column "fields" will reduce the memory consumption big time.

Login (copy)