Find corrupt VHDX profiles in Azure Files

Edit (05/24/2022): We finally discovered the cause of all of our Hosts daily stress and alerts.  Including solving this problem.  We switched to using Breadth-first load balancing from Depth-first.  This stopped piling users onto a server before moving to another one, and therefore steadily rose processing and RAM usage instead of spiking it, reducing all kinds of issues.


If you haven't had the pleasure of dealing with this, be thankful. There is some set of parameters which, when met, corrupts a user's mounted VHDX, causing them to receive messages from the OS that their disk needs to be repaired, and/or they're logged in with a temp profile. Once detected, and the user logs off and back on again, FSLogix will create a new VHDX, put it in the same directory, rename the original VHDX and append "CORRUPT" to the beginning of the filename. If you can't tell, this is bad... mmmkay? If you don't have Onedrive or some other backup enabled, the user just lost everything saved to their profile.


I have gone round and round with Microsoft Support about this problem. The conclusion of which is that when an AVD host is heavily utilized to the point of throwing error messages related to CPU or RAM usage/exhaustion, this CAN cause corruption. What the actual set of parameters is that causes this corruption is unknown or not fully understood. Microsoft's recommendation is to add more host resources so it doesn't get to the point of CPU/RAM exhaustion. Fine, fair point, but still, c'mon guys... You own AVD/FSLogix, which means this renaming logic is coded somewhere, and you don't know either?! Doubtful.


Anyways, corrupting profiles is one problem, but what about all these orphaned disks that are lying around, taking up space, and literally can't be fixed, can't be mounted somewhere else, can't be used at all. In some of our deployments, this equalled about 2-3TB of used space. At about $120/100GB/month of provisioned space, this could not be overlooked. So, I took my other script from here: https://seehad.tech/2021/08/24/searching_azure_file_share_to_match_string/ and modified it to search for CORRUPT profiles.

Leave a comment