Microsoft was undoubtedly the first to surprise everyone in the new year. An issue in Exchange's antivirus engine is causing email delivery to fail on Exchange 2019 and Exchange 2016 servers. Luckily Microsoft provided mitigation using a script while a patch is in the works.
Antivirus Engine Bug
With the new year, the usage of a signed 32-bit number is suspected to be the point of failure. Users started seeing a repeated error message:
"The FIP-FS 'Microsoft' Scan Engine failed to load. PID: 23092, Error Code: 0x80004005. Error Description: Can't convert '2201010001' to long."
With a max value of 2147483648, a 32-bit number was not enough to store the new value starting with 22 (due to the year change to 2022). This is likely why the components crashed, causing messages to get stuck. Officially, Microsoft lists that:
"The problem relates to a date check failure with the change of the new year and it not a failure of the AV engine itself. This is not an issue with malware scanning or the malware engine, and it is not a security-related issue. The version checking performed against the signature file is causing the malware engine to crash, resulting in messages being stuck in transport queues."
Identify Affected Servers
Both Exchange 2019 and 2016 can be affected, luckily, there are specific errors that indicate an issue. Both error event id 5300 and 1106 are indicators. Using Lansweeper's event log scanning, scanning these events are easily found and you can get a quick overview of servers affected.
Bug Mitigation Live
To resolve the issue, Microsoft released a script that resets the antivirus scan engine version to a new version that doesn't run into the Y2K22 issue and that will be able to automatically update in the future. On their official Exchange blog, they also listed a Q&A with a lot of relevant information in addition to steps to do perform the changes manually if you prefer going old-school.