So i cut back most of the growth to better placed buds closer to the trunk. I will cut back the lower branches more and the parent thread grafts in spring once I see the growth:
Here is a closeup of the grafts that have taken nicely. You can also see the one that did not take as it was shaded out over the growing season (the parent branches still have a bit of wire on them):
And my goal for this tree. A few options with how the apex will end up. I think I may reduce the top to fix the scare but not sure yet (pic 3):