Button Animation: Menu UI (Part IV)
Updated: Aug 22, 2019
When you set a button's transition method to "Animation", Unity automatically creates an animator with all of the default states and all of the correct transition triggers between them. This means that if you're still using the Standalone Input Module to control menu navigation, then the act of moving to a button to highlight it will put that button in its "Highlighted" state without the need for any additional hookup. Pretty nice. And for most people, that may be enough; you resize the highlighted button like they do in all of the tutorials, and that's that. In one of the tutorials I watched, they even come right out and say that most animation states will just be a single keyframe, leaving it to Unity to interpolate between them as the button transitions. The problem with this is the same one I mentioned last time, that Unity sometimes gets it wrong, and also that this really isn't "animation". For my highlighted state, I wanted a real animation, one that was a few frames long and looped when it finished. And while doing my "alien translation" thing with the button text isn't a true animation either (Unity can't interpolate between different sprites, just swap them out), I wanted to keyframe the change and implement "Translating" as a valid state machine state the buttons could be in. After a few days of trial-and-error, this is what I came up with:
The first thing to note is in the bottom right corner. This is the UI Button Controller, meaning that it applies to every and any button I create. I could create custom ones if the need arose, but this one is generic enough to work for all of them, and good thing because there are a ton of buttons in my game. The default-created states are Normal, Highlighted, Disabled, Pressed, and the special colored states Entry, Exit, and Any State. Exit is hardly ever used (which is why there are no transitions to or from it), and Entry isn't a state so much as a pointer for which of the other states to start out in when the button becomes active. I have it set to Normal for lack of a better solution; I wanted the buttons to already be translating when the menu comes up, which meant starting in either the IdleTranslate or HighlightedTranslate state (more on those later)...but without some kind of trigger or other check, I wouldn't know which one of the two. So instead I start all the buttons in Normal and immediately send the signal to move to the correct state from there. "Any State" is just shorthand to help make the diagram less cluttered. If you want to be able to move into the Highlighted state from any other state, and the condition for doing so is the same (looking for Unity's auto-created "Highlighted" trigger), then you can set that up with one arrow, as shown. I actually had to take it away from the Normal state, because my tests seemed to show that the "Normal" trigger was literally "not being in any other state", which screwed up as soon as I added the custom extra states I needed. Now you can't get back to Normal from any state, just the ones I specifically want you to be able to.
The IdleTranslate and HighlightedTranslate are six keyframes, each, swapping the button sprite every two and looping until the signal is sent that the menu is fully loaded and they can move into Normal and Highlighted states, respectively. There should absolutely be a way for these to be combined into a single Translate state, but because of the "Write Defaults" fiasco I mentioned last time, there isn't a way to keep the highlight visible in the HighlightedTranslate state without manually turning it on there. This also simplifies exiting from those two states. As I said above, when the translation finished, the state machine would look for other transitions, and if it couldn't find any, return to the Normal state. The button that was supposed to be highlighted should get the Highlighted trigger and go there instead, but sometimes it would be off by a frame (or worse, a keyframe, which runs on a different clock cycle), and you'd be able to see the highlight flashing on and off while it settled itself. My hotfix for this unwanted behavior was to remove the Any State -> Normal transition and add the SetSelected state. The only way out of HighlightedTranslate is into SetSelected, and that transition happens after a fixed time, every time. The SetSelected state itself is just a single keyframe, one which has an animation event calling a method to update current selected (remember all that I explained last time?). Doing this automatically sets the "Highlighted" trigger Unity is waiting for, and transitions into the Highlighted state correctly from there.
I think the last thing to mention is the Off state, which is a recent addition. On some of my menus, the button is not visible on launch, because the player must take additional action first to make it appear. The obvious example of this is the Player Select menu, when the Map Select button only appears after all of the registered players have made their choices, and should therefore also disappear if someone tries to back out. So the "TurnOffTrigger" is set twice in my code, whenever someone presses back on Player Select, and whenever that menu first pops up. In that latter case, the "hidden" button will start in Normal just like every other button, but receive the trigger and use the Any State -> Off transition to get where it should be. The only way out of the Off state is by getting the correct trigger (set in code when all the criteria are met), and the only place it can go is into HighlightedTranslate, since that's what we want it to be doing when it does become visible.
One challenge I faced was how to make the UI translate again whenever a menu closed. This wasn't a problem of needing more animation states or triggers, just sending the ones I had at the correct time. In fact, I do so as an animation event as part of the MenuClose animation I talked about last time. Let me leave you with this gif of a menu opening and then closing again, showing the button translations and then the glowing Highlighted animation: