Remote / Orchestrator PS commands not working - RD Connection Broker server is not available

When I run the commands while logged in I get the results immediately. This is not in a "run as administrator" session. This is a newish 2012 R2 install.

Here is from opening PS right from the desktop:

====

PS U:\> hostname
RDCB001

PS U:\> Import-Module remotedesktop

PS U:\> Get-RDServer -ConnectionBroker rdcb002.ads.local
Server                                             Roles                                                                                              
------                                             -----                                                                                              
RDCB001.ADS.LOCAL                                  {RDS-CONNECTION-BROKER}    

====

Now, from a pssession:

=====

$username = "ads\admin.acct"
$password = ConvertTo-SecureString "Mypassword" -AsPlainText -Force

$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $username, $password
$currsess = New-PSSession -ComputerName rdcb001.ads.local -Credential $cred
Enter-PSSession -Session $currsess

[rdcb001.ads.local]: PS C:\Users\admin.curtis\Documents> whoami
ads\admin.acct
[rdcb001.ads.local]: PS C:\Users\admin.curtis\Documents> hostname
RDCB001

[rdcb001.ads.local]: PS C:\Users\admin.curtis\Documents> import-module remotedesktop

[rdcb001.ads.local]: PS C:\Users\admin.curtis\Documents> Get-RDServer -ConnectionBroker rdcb002.ads.local
Get-RDServer : The RD Connection Broker server is not available. Verify that you can connect to the RD Connection Broker server.
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Get-RDServer

======

I only have 2 broker servers, and it does the same regardless of the -connectionbroker value (the example here is after making sure that the active broker is this server)

I get the same error when running this from orchestrator as a "run a command" with powershell -file c:\script\blah.ps1