Encountered this problem on an Exchange 2016 two member DAG. Microsoft was able to reproduce it in their lab environment. Updated to CU5 and disabled shadow redundancy and seems to fixed it after my hours troubleshooting:
Set-TransportConfig –ShadowRedundancyEnabled = $false
I only had to apply this on the primary node, as the problem is not replicated when I failover to the DR site.