SCB Global Ltd - Notice history

Dear SCB-Global customers and partners,

The Global WebAgent has been restored and remains stable after Microsoft completed its mitigation steps last night. We are therefore marking this incident as resolved now while we continue to observe the situation closely.

If any customer is still experiencing issues, please reach out to us via your regular support channels.

Thank you for your patience and cooperation.

Update - Update: Microsoft has completed their mitigation steps overnight, and we are now receiving confirmation that the Global WebAgent is back online and functioning as expected.

We will keep this incident under monitoring and continue to observe the situation closely, in case the issue reappears.

If any customer is still experiencing issues, please reach out to us via your regular support channels.

Thank you!

Incident Update: Our Global WebAgent URL is now accessible from some locations; however, it remains unavailable in others. We will keep this incident in Monitoring status while Microsoft continues implementing mitigation steps and will continue assisting impacted customers in restoring services via the available workaround.

Progress on Microsoft’s side can be tracked here: https://azure.status.microsoft/en-us/status

Workaround Status: Impacted customers can use the regional WebAgent URLs. This solution has proven successful, and we continue assisting customers who have requested help with its configuration. We encourage any affected customers to log a support case to apply this workaround.

Incident status: Monitoring

Incident Update: Our investigation confirms that the Microsoft outage is the root cause of this incident.

We will continue to monitor the situation closely while assisting impacted customers with transitioning to regional WebAgents to ensure service continuity.

Thank you!

Incident Update from:
We continue to investigate further to eliminate infrastructure components as a potential cause. At this point, all evidence indicates that the Microsoft outage remains the most plausible root cause. Investigation is still ongoing.

Workaround Status:
The workaround offered, configuring regional WebAgents, has proven successful. We strongly encourage impacted customers to request assistance in applying this workaround by logging a support case through the usual channels.


Microsoft Incident Status: https://azure.status.microsoft/en-us/status

Azure Front Door - Connectivity issues - Applying mitigation

Starting at approximately 16:00 UTC, customers and Microsoft services leveraging Azure Front Door (AFD) may have experienced latencies, timeouts, and errors. We have confirmed that an inadvertent configuration change was the trigger event for this issue.

Current status:
"We have initiated the deployment of our last known good configuration. This deployment was initially expected to complete within 45 minutes; however, due to protective blocks we have put in place to safeguard the AFD service, we are encountering some delays. While progress is ongoing, these safeguards are extending the overall deployment time. Once the rollout is complete, we will begin recovering nodes and re-routing traffic through healthy nodes to accelerate recovery.

Customer configuration changes will remain temporarily blocked while we continue mitigation efforts. We will notify customers once this block has been lifted.

Some customers may also have experienced issues accessing the Azure management portal. We have failed the portal away from AFD to mitigate these access issues. Customers should now be able to access the Azure management portal directly, and while most portal extensions are functioning correctly, a small number of endpoints (e.g., Marketplace) may still experience intermittent loading problems.

We do not yet have an ETA for full mitigation, but we will provide another update within 30 minutes or once the deployment has completed. At that point, we will have a clearer understanding of the timeline for full recovery."

Thank you!

Update from Microsoft: https://azure.status.microsoft/en-us/status
"Starting at approximately 16:00 UTC, customers and Microsoft services that leverage Azure Front Door (AFD) may have experienced issues resulting in latencies, timeouts and errors. We have confirmed that an inadvertent configuration change as the trigger event for this issue.

We are taking several concurrent actions: Firstly where we are blocking all changes to the AFD services, this includes customer configuration changes as well. At the same time, we are rolling back our AFD configuration to our last known good state. As we rollback we want to ensure that the problematic configuration doesn't re-initiate upon recovery.

Customers may have experienced problems accessing the Azure management portal. We have failed the portal away from AFD to mitigate the portal access issues. Customers should be able to access the Azure management portal directly, while all portal extensions are working correctly there may be a small number of endpoints that might have a problem loading (i.e. Marketplace).

We do not have an ETA for when the rollback will be completed, but we will update this communication within 30 minutes or when we have an update. "

Update:
Our ability to fully troubleshoot our infrastructure has been restored, and we are currently performing comprehensive checks. At this time, we have not identified any malfunctions within our infrastructure beyond the impact of the Microsoft outage. Investigation is ongoing.


Workaround available: As a temporary solution while services are being restored, customers can use the regional WebAgents addresses, which are currently unaffected by this issue.

Impacted customers who do not have a regional WebAgent configured and wish to apply this workaround can log a case through the support channels.

Thank you!

Incident Update:

We continue our investigation; however, progress is limited due to the ongoing Azure outage, which is impacting our ability to fully troubleshoot our infrastructure.

Workaround Available:

As a temporary solution while services are being restored, customers can use the regional WebAgents addresses, which are currently unaffected by this issue.

Update: The issue may be related to a broader Microsoft outage, as indicated on the Azure Status Page: https://azure.status.microsoft/en-us/status

We are continuing to investigate the incident to eliminate infrastructure components as a potential cause.

An update will be provided within the next 60 min

Thank you!