Earlier today we identified an issue causing exports from Procore to fail at a higher than normal rate. This is limited to the Procore Vista legacy integration.
Customers would have seen an error message "Failed to update. Please try again later." The customer could have tried again and gotten success. However, the error rates were unacceptable and so we took action.
The issue has been fixed as of 3pm CST 9/18.
If a customer has a failed export, have them try again. The data is not stuck or queued and it is safe to simply try export again.
The incident has been resolved.
We are experiencing elevated error levels since about 11am CST that is impacting legacy services and causing performance delays. We are actively investigating the issue.
We have identified the issue and determined that it is limited to the Procore <> Vista legacy integration.
This incident is now limited to the Procore <> Vista legacy integration.
The Viewpoint services for the Procore <> Vista integration are failing to complete at a higher than normal rate. It is not all customers and it is not every hourly run, but the failure rate is higher than normal. Customers would experience a delay in data transfer to Procore and a higher than normal failure rate when exporting (the error may say something generic like failed entirely).
We expect some database maintenance to complete overnight. We will continue to monitor the situation. We plan to leave this incident open and provide an update in the morning.
Maintenance completed around 6:51pm late yesterday. Legacy Viewpoint syncs for the Procore <> Vista integration started working normally after that. They ran overnight and through this morning as expected.
This incident has been resolved.
There is an issue preventing processing of workflows. We have identified and addressed the issue. We are continuing to monitor, but need to work through the queue. We will post an update as soon as we have more info.
Monitoring We are working down the queue, currently expect to return to normal in 1-2 hours. We are continuing to monitor for any further issues.
This incident has been resolved.
We have identified the issue and the impact is minimal to end customers, but not resolved to our high standards. We are keeping this incident open to review in the morning.
Since 5:17am CST 6/25 there have been an abnormal number of schedules that finish a service, but remain in a started status. It will then appear as "Cancelled" in the job logs after a period of time.
We will continue to monitor, and provide updates tomorrow morning as we have them.
Some integrations with file transfers, most notably the Procore <> Vista Financials integration are failing to upload to our file share. We are investigating and will provide status updates as we have them.
This is impacting the hourly syncs for this integration since yesterday around 5pm CST. This would also be impacting some Concur services or others that are doing uploads from Vista.
We have identified the issue and put a fix in place around 10:15am CST. We are monitoring the services to ensure scheduled runs complete as expected. Customers would have experienced a delay in data transfer from Vista to other system (Procore, Concur, etc.) from 5:22pm CST yesterday to 10:15am CST today.
This incident has been resolved.
We are actively working to resolve an issue causing integration schedule runs to be delayed by up to 30 minutes.
The issue has been identified and mitigated. Schedules are executing as expected. We are monitoring the platform. Customers would have experienced a delay in data transfer.