I've just started using KStars/Ekos, but I have certainly already hit this problem.
My first *batch* of attempts to use the polar alignment assistant were just as you are describing: while the yellow hint line is supposed to represent the desired adjustment in Altitude and the green line Azimuth, it was obvious that altitude adjustments were "moving" the FOV along the green line and vice versa.
Usually such issues, when the telescope driving software seems confused about the directions, tells you it's moving one way but actually goes another and so on, are indicative of the fact the driver is confused about where and how the telescope is pointed.
The "how" part was what I noticed was wrong in my case: the Side of Pier reported by the driver was not matching the side of my tripod the telescope was on.
Unfortunately, in my case (Takahashi Temma driver) it is impossible to just tell the driver to change the side (I believe there's an open thread on this forum about that), so the way to go for me was to physically move the telescope onto the side the driver thought it was on, re-solve the position and re-attempt the alignment procedure.
From that point the adjustment lines provided by the polar alignment assistant were matching the actual axes the are supposed to represent.
However, I still never managed to harness the polar alignment routine in Ekos, because even after figuring out the situation described above, adjusting the mount according to the provided instructions were increasing my polar alignment error. Quite literally: I record the calculated error, adjust as the assistant is telling me, re-run the procedure, and the error is like twice as much.
I ended up resorting to the good ol' drift alignment method via PHD2, which worked like it should.
Full disclosure: later on I realized that my GPS dongle was not working (still not sure what's wrong with it, but neither gpsmon nor cgps were showing any fix), so that could have been at least part of the problem. I noticed that after having already aligned using the drift method, and switched the EKos settings to using manual time/location and to Ekos providing them to all equipment, and at least driving the scope around the sky, as well as doing guided exposures worked fine from then on.
My point being: confirm whether the telescope is where (in the cumulative sense of location, time, side of pier and, obviously, the exact location in the sky) the driver thinks it is.
Good luck!