Microsoft rolled out emergency fix for Y2k22 bug in Exchange servers

Microsoft released an emergency patch to fix the Y2k22 bug that is breaking email delivery on on-premise Microsoft Exchange servers.

Microsoft has rolled out an emergency fix that addresses the Y2k22 bug that is breaking email delivery on on-premise Microsoft Exchange servers since January 1st, 2022.

“We have addressed the issue causing messages to be stuck in transport queues of on-premises Exchange Server 2016 and Exchange Server 2019. The problem relates to a date check failure with the change of the new year and it not a failure of the AV engine itself. This is not an issue with malware scanning or the malware engine, and it is not a security-related issue.” reads the post published by Microsoft. “The version checking performed against the signature file is causing the malware engine to crash, resulting in messages being stuck in transport queues.”

The problem is caused by a bug in the FIP-FS anti-malware scanning engine. FIP-FS is the anti-malware scanning engine used by Microsoft to protect its users, it was used starting with Exchange Server 2013. The security researcher Joseph Roosen explained that the root cause of the issue is the use of a signed int32 variable to store the value of a date, which has a maximum value of 2,147,483,647.

This means that dates related to 2022, having a minimum value of 2,201,010,001 or larger, can be stored in the signed int32 variable. The scanning engine fails to handle the date and generates an 1106 error as visible in the Exchange Server’s Event Log.

Log Name: Application 
Source: FIPFS 
Logged: 1/1/2022 1:03:42 AM 
Event ID: 5300 
Level: Error 
Computer: server1.contoso.com
Description: The FIP-FS "Microsoft" Scan Engine failed to load. PID: 23092, Error Code: 0x80004005. Error Description: Can't convert "2201010001" to long.

Log Name: Application 
Source: FIPFS 
Logged: 1/1/2022 11:47:16 AM 
Event ID: 1106 
Level: Error 
Computer: server1.contoso.com 
Description: The FIP-FS Scan Process failed initialization. Error: 0x80004005. Error Details: Unspecified error.

The fix released by Microsoft is temporary, it addresses the problem while the IT giant is working on a final update.

Microsoft released a PowerShell script (‘Reset-ScanEngineVersion.ps1’) that could be executed to stop the Microsoft Filtering Management and Microsoft Exchange Transport services. It also deletes older AV engine files, download the new AV engine and restart the services.

Before running the script, admins have to change the execution policy for PowerShell scripts by running Set-ExecutionPolicy -ExecutionPolicy RemoteSigned.

“Implementation of the solution requires customer actions, and it will take some time to make the necessary changes, download the updated files, and clear the transport queues. Actions can be automated with the scan engine reset script from https://aka.ms/ResetScanEngineVersion or they can be performed manually. Whether you perform the steps automatically or manually, they must be performed on every on-premises Exchange 2016 and Exchange 2019 server in your organization. If you use the automated script, you can run it on multiple servers in parallel.” states the post.

Microsoft has also provided the step-by-step procedure to update the scanning engine manually. The company states that the new AV scanning engine will have version number 2112330001:

“The version of the updated scan engine starts with 2112330001; is this right? Should we be concerned that it seems to reference a date that does not exist?
The newly updated scanning engine is fully supported by Microsoft. While we need to work on this sequence longer term, the scanning engine version was not rolled back, rather it was rolled forward into this new sequence. The scanning engine will continue to receive updates in this new sequence.”

Follow me on Twitter: @securityaffairs and Facebook

[adrotate banner=”9″]

[adrotate banner=”12″]

Pierluigi Paganini

(SecurityAffairs – hacking, Y2k22 bug)

[adrotate banner=”5″]

[adrotate banner=”13″]