HTTP Status 400 from TDA

by **analyticdelta** » Thu Dec 21, 2017 6:18 pm

Hi Mark,

Two days ago I successfully downloaded transactions for a TD Ameritrade retail account. The next day, yesterday, according to the FM Pro logs, I receive an HTTP Status 400 with the error message: "The request sent by the client was syntactically incorrect.". The problem persists today. Have you heard of others having errors with TDA transaction retrieval in the last couple days? Could it be something I am doing?

by **Mark** » Thu Dec 21, 2017 7:40 pm

Hi analyticdelta,

The TD Ameritrade interface is working okay from here, and we haven't had other problem reports. Do you have the latest transaction module (232)? Are you able to retrieve from other broker interfaces if you have them? Maybe you can email me the "ofxretr.txt" and "ofxpost.txt" log files right after a retrieve failure.

by **analyticdelta** » Thu Dec 21, 2017 9:11 pm

Hi Mark,

Thanks for your prompt reply.

This is a weird one.

The contents of ofxretr.txt is:

Code: Select all: <html><head><title>VMware vFabric tc Runtime 2.9.3.RELEASE/7.0.42.A.RELEASE - Error report</title><style></style> </head><body><h1>HTTP Status 400 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The request sent by the client was syntactically incorrect.</u></p><HR size="1" noshade="noshade"><h3>VMware vFabric tc Runtime 2.9.3.RELEASE/7.0.42.A.RELEASE</h3></body></html>

I run Windows in a VMWare Fusion VM instance hosted by OS X Sierra.
I wonder if FM posted its http request somehow to the VWWare vFabric tomcat instance instead of TDA's OFX server due to some network configuration weirdness in the VM? Surely this error message is not coming from TDA. Ever seen this?

Here are the redacted contents of ofxpost.txt:

Code: Select all: OFXHEADER:100 DATA:OFXSGML VERSION:102 SECURITY:NONE ENCODING:USASCII CHARSET:1252 COMPRESSION:NONE OLDFILEUID:NONE NEWFILEUID:NONE <OFX> <SIGNONMSGSRQV1> <SONRQ> <DTCLIENT>20171222011147.187 <USERID>[myredactedusername] <USERPASS>[myuserpass] <LANGUAGE>ENG <FI> <ORG>Ameritrade </FI> <APPID>QWIN <APPVER>2600 </SONRQ> </SIGNONMSGSRQV1> <SIGNUPMSGSRQV1> <ACCTINFOTRNRQ> <TRNUID>3406-FM7-20171222011147.187-187 <ACCTINFORQ> <DTACCTUP>19700101000000 </ACCTINFORQ> </ACCTINFOTRNRQ> </SIGNUPMSGSRQV1> </OFX>

One other item possibly of note: I initially installed FM the typical way. Then I read about Portable mode and set the data location to reside on a shared network volume. Then I uninstalled FM and reinstalled FM to the same shared network volume directory so that I can run FM, data and exe, from there. That works fine, but I do note that I think the problem started after that.

p.s. It might be nice (advisable) if FM redacted the user id too in the ofxpost.txt log file.

by **Mark** » Fri Dec 22, 2017 10:08 am

Hi analyticdelta,

I've not run into this before. I do see mention of "VMware vFabric tc Runtime 2.9.3.RELEASE" in that error response, so it does appear to be coming from there, and not TD's OFX server. If you have access to a regular Windows PC you might try it on there, to make sure everything else is okay, but I would agree it seems some config issue with that VM. You might also test IE on your VM, and make sure it is functioning properly. FM and IE use the same windows component (WININET) to access the internet securely.

Thanks for the feedback on possibly redacting the username from the log file. We chose to leave it in there, and just always redact the password. This can help in debug situations sometimes, and if you'd rather not leave log files on your system with your username in them, you can turn off logging at "Options / General Preferences... / Other / Create Log Files When Importing Transactions".

by **analyticdelta** » Fri Dec 22, 2017 11:33 am

Actually, I was able to determine that the error message actually is coming from TD Ameritrade. Their ofx server uses VMWare vFabric tomcat as its webserver. I had mentioned VMWare because I use their Fusion product, but turns out VMWare vFabric is a well known server virtualization technology that large corps use to host sites. You can verify that such an error response comes directly from TD Ameritrade by trying to load their ofx server url, https://ofxs.ameritrade.com/cgi-bin/apps/OFX, in your web browser and you'll see that such an empty http request via the browser generates the exact same message.

By the way, my IE, and all other networked apps, work fine on the VMWare Fusion VM instance so that is further proof that use of VMWare Fusion was a red-herring.

Will be interested to know your insights/suggestions. Perhaps TD Ameritrade changed something?

At least we now know that the error response is coming from TDA.

by **Mark** » Fri Dec 22, 2017 11:48 am

Hi analyticdelta,

Interesting, I too see the same thing when going to that link. Are you using the latest transaction module 232?

Can you try this experiment:

Open up the "File / New Portfolio Wizard..." and enter any fake portfolio name, and leave the account number blank. Press Next. Choose "TD Ameritrade" and for the login use "johndoe" and "12345", and press Next. Choose the default of retrieving transactions/positions, and press "Next". Do you get a "Signon invalid" error message?

I did the above test here, and that is what I got. I'm just trying to see if there is something specific to your existing account, or whether it is your whole system.

by **analyticdelta** » Fri Dec 22, 2017 12:16 pm

Mark, I did the experiment you suggested and yes, it did return the Signon Invalid error dialog.

As per the transaction module, I looked under Help > About and it said 531. I updated to 532 and reran and received the same error as the original.

by **Mark** » Fri Dec 22, 2017 12:26 pm

Hi analyticdelta,

Thanks. Can you try that same experiment (leaving account # blank again), but use your same login details you are using on the problematic account? If it works, just cancel the setup, so you don't create another sub-portfolio.

by **analyticdelta** » Fri Dec 22, 2017 12:33 pm

Done. I received the same error except the account id was missing in the error dialog (of course).

by **Mark** » Fri Dec 22, 2017 12:36 pm

By "same error", you mean the "HTTP Status 400" error, right? You didn't get the same "Signon invalid" error as when using a fake login, right?

by **analyticdelta** » Fri Dec 22, 2017 12:37 pm

Correct.

by **analyticdelta** » Fri Dec 22, 2017 12:38 pm

As a side note, out of interest, is FM a Delphi app? I see the DBIASM dbs per security and wondered. If so, use Embarcadero?

by **Mark** » Fri Dec 22, 2017 1:09 pm

I'm not sure what is wrong here, and is difficult for me to debug, as I can't reproduce the problem here. I've also tested it with a valid TD login, and it is working here, so it seems something specific to your setup or login. To debug further, I can think of 3 ideas:

1) Test it on another computer, running plain Windows. Maybe something odd is happening related to VMware.

2) Test it with Quicken

3) Call TD, and ask them for help explaining why their server is returning that error. I suspect they won't help you very much without testing it under Quicken first, and verifying the error happens there as well.

No, FM isn't a Delphi app. It is a native Win32 API app. I don't know what DBIASM is...

by **analyticdelta** » Fri Dec 22, 2017 1:13 pm

Thanks for trying Mark. I will check out some of those options. (The dat files just are associated, on my system, with a dbiasm client app, so I thought maybe that was their underlying structure. My mistake. It sure is a nice app, good job with it.)

by **analyticdelta** » Fri Dec 22, 2017 6:07 pm

I created another portfolio for another tda account I have and that one worked fine (different account #, different login). Perhaps TDA blacklisted the other account after making too many OFX requests or something for some security reason. As you'll recall it worked fine the first time. I won't know until I contact them after Christmas. Just thought you would be interested to know. It doesn't have anything to do with VMWare. I also installed a trial of Investment Account Manager and the account that I am having trouble with received an invalid signon and, like FM, the other account worked fine.

As an aside, for kicks, I used Wineskin Winery to bottle FM to run on Mac under Wine without VMWare Fusion VM and everything I tried worked fine (save the same account issue which is apparently something at TDA with that account). For the record, Investment Account Manager ran under Wine fine as well. These older Win32 C++ apps often work well under Wine as they do not require .NET libraries. It is a nice option for Mac users because it means they don't have to purchase a commercial VM like Parallels or VMWare to run your app.

HTTP Status 400 from TDA

Who is online