@justinball
I had to do a little research, so I may be way off. I apologize ahead of time if I'm sending you down the wrong path.
After all my research into what displayHeight and displayWidth were, I came back to the first thing I thought of -- that it's inside an iframe. It's actually the first thing that comes to my mind anyone questions the sizing of LTI tools that aren't opened in their own window/tab.
The introduction to the Content Items specification says:
The Content-Item Message allows a Tool Provider (TP) to be involved in the process of adding items to a content page within a Tool Consumer (TC). The TP is launched by the TC with the ContentItemSelectionRequest
message type allowing the TP to provide the user with an interface for selecting one, or more, items for placement into the TC. This JSON binding describes the format used to return the selected items back to the TC.
That makes it sound like the LTI (tool provider TP) is the one that uses displayHeight and displayWidth, not Canvas (tool consumer TC). One way of testing this might to be to edit the iframe tag in Canvas to be larger than than the displayHeight and displayWidth and see if they come out smaller than the iframe element. That would be up to you as a tool provider to specify the size, which you may or may not have done.
Regardless of how that really works in the background, I think the root issue is that Canvas embeds the LTI tool inside an iframe. That iframe is created when the content is put on the page and not when the content is launched. That iframe acts as a sandbox and things inside the iframe can't manipulate parent items, which is where the size of the iframe is specified, and is unable to change the width and height on the iframe.
The 300x150 window is telling Canvas how big to make the iframe and it can launch an LTI to go into the space, but beyond that, it is unaware of what goes inside that iframe. For security purposes, the LTI can't directly manipulate the actual size of the iframe because it's from a different domain. Unless the content inside the iframe is loaded from the same domain (including hostname) as the Canvas page itself, it becomes subject to the Same-origin policy and the two windows (Canvas as the parent window and the LTI tool in the iframe) have very limited ability to communicate with each other. Note that window.document is not one of the things shared.
Window.postMessage() is the recommended way to have communicate between documents from different domains. H5P recommends/supplies a window resizer script for use with Canvas. In either case, you need to make the parent window aware of what's happening by placing custom JavaScript into Canvas and you need to make the child window (LTI) talk to the parent by placing JavaScript in the embedded content.