RDS Farm setting applied through GPO causing connection issues

I'm having a strange issue with some RDS GPO's. Since automating the RDS server build process through SCCM I've moved the RDS farm / connection broker settings to a GPO. This works fine, once the servers are built the GPO applies and the machines join the farm.

However, when any GPO associated with the farm is changed, all the sessions connected to the farm are dropped. User can reconnect but this is very inconvenient for us to try and push shortcuts and what not during production.

After searching fro a while, to me, it seems like the GPO that assigns the farm settings is being reprocessed, causing the settings to drop and connections to be dropped.

I wouldn't think that this would be normal behavior? Can anyone confirm for me that this should work as I expect it to?

Thanks!