Interaction System from The Lab

Transcription

1 Interactin System frm The Lab The Interactin System is a series f scripts, prefabs and ther assets that were the basis f all the minigames and ther scenes in The Lab. The system was designed specifically t be lightweight and flexible s that it culd supprt all the experiments that we were wrking n at the time. Mst f the included cmpnents were actually used in The Lab. We have als included a few cmpnents that we didn't end up using in The Lab but yu might still find useful. Hpefully it can serve as a gd starting pint fr yur wn experiments and help yu in develping yur prjects.

2 Getting started Interactin System dcumentatin - Create a new scene - Delete the Main Camera bject - Drag in the Player prefab frm Cre/Prefabs int yur scene. This prefab sets up the main Player cmpnent and the hands. It als hks int all the relevant SteamVR things needed. - Yu shuld be able t see the scene in the headset nw alng with yur cntrllers being tracked in the scene. - Add the Interactable cmpnent t any bject in the scene. All the ther cmpnents n this bject will then start receiving relevant messages frm the player s hands. (Lk at Samples/Scripts/InteractableExample.cs fr example usage) - We have included a few cmmnly used interactable classes such as the Thrwable. Adding this cmpnent t yur bject will allw it t be picked up and thrwn by the player. - T add teleprting t yur scene drag in the Teleprting prefab frm Teleprt/Prefabs int yur scene. This will set up all the teleprt lgic. - Drag in sme TeleprtPint prefabs frm Teleprt/Prefabs t add lcatins that the player can teleprt t. - Yu can als add the TeleprtArea cmpnent t any bject in yur scene. This will allw the player t teleprt anywhere alng the cllider fr that bject. With these basic building blcks yu can then g n and create sme fairly cmplex bjects. Fr sme examples please take a lk at the Interactins_Example scene in Samples/Scenes.

3 Sample scene The sample scene Interactins_Example in the Samples/Scenes flder includes all f the majr cmpnents and is a gd place t familiarize yurself with the system. The scene cntains the fllwing elements: - Player: The Player prefab is at the cre f the entire system. Mst f the ther cmpnents depend n the player t be present in the scene. - Teleprting: The Teleprting prefab handles all the teleprting lgic fr the system. - InteractableExample: This shws a very simple interactable that shws the basic aspects f receiving messages frm the hands and reacting t them. - Thrwables: These shw hw t use the interactin system t create a slightly mre cmplex bject that can actually be used in a game. - UI & Hints: This shws hw hints are handled in the interactin system and hw it can be used t interact with Unity UI widgets like buttns. - LinearDrive: This is a slightly mre cmplex interactable that cmbines a few different peices t create an animated bject that can be cntrlled by simple interactins. - CircularDrive: This shws hw interactins can be cnstrained and mapped differently t cause mre cmplex mtins. - Lngbw: This is the actual Lngbw used in The Lab. It is ne f the mre cmplex bjects that we've created using this system and shws hw simple peices can be cmbined t create an entire game mechanic. Ging ver the different bjects in this sample scene shuld give yu a gd idea f the breadth f the interactin system and hw t cmbine its different parts t create cmplex game bjects.

4 Dcumentatin Interactin System dcumentatin This dcumentatin will g ver sme f the basic cmpnents included in the interactin system. The system is brken up int a few different parts: Cre: - At the cre f the interactin system are the Player, Hand and Interactable classes. The prvided Player prefab sets up the player bject and the SteamVR camera fr the scene. - The interactin system wrks by sending messages t any bject that the hands interact with. These bjects then react t the messages and can attach themselves t the hands if wanted. - T make any bject receive messages frm the hands just add the Interactable cmpnent t that bject. This bject will then be cnsidered when the hand des its hvering checks. - We have als included a few cmmnly used interactables such as the Thrwable r the LinearDrive. - The Player prefab als creates an InputMdule which allws the hands t mimic muse events t easily wrk with Unity UI widgets. - The interactin system als includes a fallback mde which allws fr typical first-persn camera cntrls using the keybard and muse. This als allws the muse t act like ne f the player's hands. This mde is particularly useful when nt everyne n the team has access t VR headset. Player - The Player class acts like a singletn which means there shuld nly be ne Player bject in the scene. - The player itself desn't d much except fr keep track f the hands and the hmd. - It can be accessed glbally thrughut the prject and many aspects f the interactin system assume that the Player bject always exists in the scene. - It als keeps track f whether yu are in VR mde r 2D fallback mde. - Using the accessrs thrugh the Player class allws the ther cmpnents t functin similarly withut knwing if the VR headset r muse/keybard is being used. - The 2D fallback mde is useful but has its limitatins. We mainly used this mde fr testing ut very simple interactins that nly required 1 hand and the trigger buttn. It was mainly

5 useful during develpment when nt everyne n the team had a VR headset n cntrllers with them at all times. - The Player als includes a few useful prperties: hmdtransfrm: This will always return the transfrm f the current camera. This culd be the VR headset r the 2D fallback camera. feetpsitinguess: This guesses the psitin f the player's feet based n the where the hmd is. Since we dn't actually knw the psitin f their feet, this can be pretty inaccurate depending n hw the player is standing. bdydirectinguess: This is similar t the feetpsitinguess in that it can be inaccurate depending n hw the player is standing. - Nte: The player class is set up t use icns t shw the feet and hands in the editr scene view but due t the way Unity wrks these icns have t be lcated in a specific flder t wrk. These icns are prvided under Cre/Icns. Mve them t a flder named "Gizms" in the rt f yur prjects Asset tree and they shuld wrk. - The 2D fallback mde can be useful during testing but it s prbably nt smething that yu want t ship with yur finished game. There are 2 ways t disable it: Uncheck the "Allw Tggle T 2D" bl n the player bject in yur scene befre making yur build. Add "HIDE_DEBUG_UI" t the list f Scripting Define Symbls in yur prject's PlayerSettings. This will nly disable the 2D debug view in builds f yur game while allwing yu t keep using it in the editr. Hand - The Hand class des mst f the heavy lifting fr the interactin system. - The Hand checks fr bjects (Interactables) that it is hvering ver and sends them messages based n the current hver state. - The hand can nly hver n 1 bject at a time and nly 1 hand can hver n an bject at the same time. - Objects can be attached t and detached frm the Hand. Only ne bject can be the bject in fcus f the hand but multiple bjects can be attached t the hand at the same time. - Once an bject is detached frm the hand then the previus bject attached t the hand (if it is still attached) becmes the bject in fcus n the hand - When nthing is attached t the hand it will always shw the cntrller.

6 - The attached bject can set AttachmentFlags that determine the behavir f the hand and the bject nce it has been attached. - The hand can be lcked frm hvering ver ther bjects r any bject depending n the situatin. - These are the messages the hand sends t bjects that it is interacting with: OnHandHverBegin: Sent when the hand first starts hvering ver the bject HandHverUpdate: Sent every frame that the hand is hvering ver the bject OnHandHverEnd: Sent when the hand stps hvering ver the bject OnAttachedTHand: Sent when the bject gets attached t the hand HandAttachedUpdate: Sent every frame while the bject is attached t the hand OnDetachedFrmHand: Sent when the bject gets detached frm the hand OnHandFcusLst: Sent when an attached bject lses fcus because smething else has been attached t the hand OnHandFcusAcquired: Sent when an attached bject gains fcus because the previus fcus bject has been detached frm the hand - These are the messages that the hand sends t its child bjects: OnHandInitialized: Sent when the hand first gets initialized by assciating itself with the device ID f a SteamVR tracked cntrller OnParentHandHverBegin: Sent when the hand starts hvering ver smething OnParentHandHverEnd: Sent when the hand stps hvering ver smething OnParentHandInputFcusAcquired: Sent when the game windw gains input fcus OnParentHandInputFcusLst: Sent when the game windw lses input fcus - These members deal with attaching and detaching: AttachObject: Attaches the bject frm the hand using the passed in AttachmentFlags DetachObject: Detaches the bject frm the hand and ptinally restres it t its riginal parent currentattachedobject: This returns the in-fcus attached bject n the hand, if any - The Hand als has a few useful prperties and functins that can be used t custmize its behavir: OtherHand: This is the ther hand n the player. This can be useful fr bjects that need t interact with bth hands such as the lngbw. HverSphereTransfrm and Radius: This can be used t custmize the hver range f the hand. HverLayerMask: This can be changed s that the hand nly hvers ver bjects in certain layers. HverUpdateInterval: The hvering check can be dne mre r less frequently depending n the requirements f yur game.

7 HverLck/Unlck: This is used t make the hand nly hver ver a certain bject. Passing in null will make the hand nt hver ver anything while it is hver lcked. This technique is used t make the hand nt hver ver bjects while the teleprt arc is active. GetStandardInteractinButtn/Up/Dwn: These are used t check the state f the trigger n the cntrller. This is useful fr simple bjects that can als be used with the 2D fallback hand. In the 2D fallback case the left-click acts as the standard interactin buttn and the bjects behave the same. GuessCurrentHandType: This uses sme SteamVR functins t figure ut which hand is leftmst and which is rightmst. GetAttachmentTransfrm: Objects can use "attachment transfrms" n the hand t figure ut hw t snap n t the hand. These are just named children f the Hand bject. The Player prefab cntains "Attach_CntrllerTip" as an example. Interactable - The Interactable class is mre f an identifier. It identifies t the Hand that this bject is interactable. - Any bject with this cmpnent will receive the relevant messages frm the Hand. - Using just these 3 cmpnents yu shuld be able t create many different and cmplex interactive bjects. - A few cmmnly used interactable system cmpnents that shw hw the system can cmbine these basic mechanics t create mre cmplicated bjects have been prvided: Thrwable - This is ne f the mst basic interactive bjects. - The player can pick up this bject when a hand hvers ver it and presses the interactin buttn (trigger). - The bject gets attached t the hand and is held there while the buttn is pressed. - When the trigger is released then any velcity that was in the hand is given t thrwn bject. - This lets yu create basic bjects that can be picked up and thrwn.

8 LinearDrive - This allws an bject t be mved by the hand between a starting and ending psitin. - The bject's current psitin is used t set a LinearMapping. CircularDrive - This allws an bject t be mved by the hand in a circular mtin. - The bject's current psitin is used t set a LinearMapping. LinearMapping - This is a number that is set by a LinearDrive r CircularDrive. - The mapping can be used t map simple hand interactins int mre cmplex behavirs. An example f this is string in the Lngbw which uses a LinearMapping t map the pulling f the bw string t the lngbw pull-back animatin. - The mapping is used by several ther classes t interplate their prperties LinearAnimatin LinearAnimatr LinearBlendShape LinearDisplacement HapticRack VelcityEstimatr - This class is useful fr estimating the velcity and acceleratin f an bject based n a change in its psitin. - In mst cases yu wuld get mre accurate results if yu get the velcity and acceleratin frm the actual cntrller but smetimes that isn't pssible such as when using the 2D fallback hand. IgnreHvering - This can be added t an bject r specific cllider if yu want it t be ignred by the hand when ding its hvering checks.

9 UIElement - Adding this cmpnent t an existing UI widget will make it s that the hands can interact with it. - This will generate muse hver and click events based n hand interactins and send them thrugh the Unity event system t wrk with existing UI widgets. - In additin it will als generate an OnHandClick event which will als pass in the hand that clicked the element. - Anther big part f the interactin system is the cncept f an ItemPackage ItemPackage - An ItemPackage is cllectin f bjects that are meant t temprarily verride the functinality f the hand. An example f this is the Lngbw. While the Lngbw is attached t the hand it srt f takes ver the base functinality f the hand. - ItemPackages have the cncept f being able t be picked up and put back where they were picked up frm. - Once picked up they remain attached t the hand until they are put back. N buttn needs t be held fr them t remain attached t the hand. The hand still passes alng messages like nrmal but these bjects usually disable sme f the base functinality f the hand, such as hvering while they are attached. Other examples f an ItemPackage frm The Lab wuld be the balln tl r the Xrtex drne cntrller. Bth these bjects take ver the hand's base functinality while they are attached. - An ItemPackage can be 1 r 2 handed. ItemPackageSpawner - This handles the lgic fr when t spawn and put away the ItemPackage and hw t attach the items t the hand nce spawned. - It als handles shwing the preview f the item r the utline f the item when it is picked up. ItemPackageReference - This cmpnent can be added t item t indicate that it is a part f an item package.

10 - There are a few ther helper classes included as a cre part f the interactin system. PlaySund - This class allws AudiClips t be played back with many mre parameters - It can take in many AudiClips and play back 1 at randm each time - It can als randmize hw the clip is played back SundPlayOneSht - This class is specifically fr sunds that nly play nce and dn't lp r need t be paused while playing Util - This is a class full f small utility functins that are used thrughut the interactin system. InteractableHverEvents - This class generates UnityEvents when it receives messages frm the hand. InteractableButtnEvents - This class translates cntrller buttn input int UnityEvents. CmplexThrwable - This class attaches bjects t the hand using a physics jint instead f simple parenting. - This allws fr mre physics based interactins with the bject nce it is attached. - Nte: This class is a little experimental. Since we didn't actually use it in The Lab it might nt be feature cmplete and may be buggy.

11 DistanceHaptics - Triggers haptic pulses based n a distance between 2 transfrms. Player (Prefab) - This is the single piece f the interactin system that cmbines all its basic parts. - This prefab arranges the player and hands in a way t make them all accessible easily. - It als cntains all the setup fr SteamVR and the 2D fallback system - Mst f the ther cmpnents f the interactin system depend n the player and sme f them assume that the player and hands are set up in this way. - There shuld nly be 1 f these in a scene. BlankCntrller (Prefab) - This is used by the Hand when it has nthing else attached. - The render mdel fr the cntrller is laded thrugh SteamVR and all its parts are articulated.

12 Teleprt - The teleprt system frm The Lab supprts teleprting t specific teleprt pints r a mre general teleprt area. - The imprtant classes are Teleprt, TeleprtPint and TeleprtArea. - All the functinality is wrapped up in the Teleprting prefab in Teleprt/Prefabs. This prefab includes all the lgic fr the teleprt system t functin. - Add TeleprtPints r TeleprtAreas t the scene t add spts where the player can teleprt t. Teleprt - This class handles mst f the lgic f teleprting. - When the tuchpad is pressed, the teleprt pinter shws up. If the pinter is pinting at a valid spt when the tuchpad is released then the player teleprts. Yu can als press 'T' n the keybard while in 2D fallback mde t bring up the teleprt pinter. - There is a slight fade t black when the player teleprts and then the game fades back in. - This class keeps track f all the teleprt markers in the scene and infrms them t fade in/ut depending n the state f the teleprt pinter. - In certain situatins it can be useful t have a separate mesh fr the flr the scene that is different frm the teleprt mesh. In these situatins the teleprt system will trace dwn frm where it hit the teleprt mesh and try t place the player n the flr mesh. The pint f this is t try t match the visual flr in the scene with the physical flr in the player's play area. - There are a few prperties that will prbably need t be tweaked: tracerlayermask: This is all the layers that the teleprt pinter will try t hit flrfixupmask: The layer that the flr is n. flrfixupmaximumtracedistance: The maximum distance t trace t try t lk fr the flr. ShwPlayAreaMarker: This tggles whether t shw the rectangle f the player's play area while teleprting. This can help in rienting the players in their physical space. arcdistance: Hw far the teleprt arc shuld g. Increasing this number will allw the player t teleprt further in the scene. This value will prbably need t be tweaked fr each scene.

13 TeleprtMarkerBase - This is the base class fr all the teleprt markers. - It cntains methds that the Teleprt class expects t be present in all the teleprt markers. - Yu can use this as yur base class t create a new type f teleprt marker. - A teleprt marker can be lcked r unlcked. The player cannt teleprt t lcked markers. TeleprtArea - This is a teleprt area that is made up f a mesh. - When teleprting nt these, the player will teleprt exactly where they are pinting (plus the flr fixup) - Add this cmpnent t any bject with a cllider and a mesh renderer t allw the player t teleprt n it. TeleprtPint - This is a teleprt pint that the player can teleprt t. - When teleprting nt these, the player will teleprt at the rigin f the pint regardless f where n the pint they were pinting. - These pints can be named - The pints als have the ability t teleprt players t new scenes. (This isn't fully functinal since yu will have t hk it up t yur scene lading system.) TeleprtArc - This draws the arc fr the teleprt pinter and des the physics trace fr the teleprt system. AllwTeleprtWhileAttachedTHand - By default yu can't teleprt using a hand that has smething attached t it. Adding this cmpnent t an attached bject bypasses that rule. - This is used by the BlankCntrller and lngbw Arrw bjects s that the player can teleprt even while they are attached t the hand.

14 IgnreTeleprtTrace - Adding this t an bject with a cllider will allw the teleprt trace t pass thrugh it. - A different way f dealing with this wuld be t put that bject n a different layer that the TeleprtArc desn't check against. - This is used n the lngbw Arrw t allw the teleprt trace t pass thrugh the arrw tip. Teleprting (Prefab) - This prefab sets up the entire teleprt system. - Dragging this int yur scene will give yu the ability t bring up the teleprt pinter in yur game. - All f the visuals and sunds f the teleprt system can be changed by mdifying the prperties f this prefab. TeleprtPint (Prefab) - Add these t yur scene t add lcatins that the player can teleprt t. - Nte: The names f sme f the bjects in this scene are hard-cded and sme f the cde will need t be mdified if yu want t change the mdels.

15 Hints - The hint system shws hints n the cntrllers. - The hints are set up in a way where each buttn n the cntrller can be called ut separately. - There is als the ability t shw text hints assciated with each buttn. CntrllerButtnHints - The hints are set up based n the render mdel f the cntrller. - SteamVR prvides a mapping frm render mdel cmpnents t buttn IDs. This mapping is used t figure ut what part f the cntrller crrespnds t which buttn. - Once a hint fr a buttn is activated, that buttn will keep flashing n the cntrller mdel until the hint is turned ff. - Hints can be fr buttns nly r with an ptinal text hint assciated with the buttn. - There are a few static methds that are used t interface with the hint system: ShwButtnHint: Flashes the specified buttn n the specified hand. HideButtnHint: Stps flashing the specified buttn n the specified hand. HideAllButtnHints: Stps flashing all the buttns n the specified hand. IsButtnHintActive: Checked if a specified buttn is flashing n the specified hand. ShwTextHint: Shws a text hint with the passed in string assciated with the specified buttn n the specified hand. HideTextHint: Hides the text hint fr the specified hand n the specified buttn. HideAllTextHints: Hides all the text hints that are currently active n the specified hand. GetActiveHintText: Gets the active hint text fr a specified buttn.

16 Lngbw - The Lngbw is an example f a cmplex game mechanic created using the interactin system. - The versin included here is exactly the same ne that we shipped in The Lab, including all the mdels, materials and sunds. - The Lngbw is built using the ItemPackage system. It is cmprised f the LngbwItemPackage prefab which spawns the Lngbw prefab in the main hand and the ArrwHand prefab in the ther hand. - The ArrwHand prefab then spawns a new arrw in the hand every time ne is fired. - All f the bw and arrw lgic is present in the fllwing scripts: Lngbw - It handles the lgic f hw the bw cntrls in ncked and un-ncked mdes - It als keeps track f hw far the bw string is pulled ArrwHand - Handles ncking and firing the arrw based n its psitin and the cntrller buttns - Handles spawning an arrw in the hand when needed Arrw - The actual arrw that gets fired - This script handles all the in-flight lgic fr the arrw including cllisin detectin and deciding when t stick in t targets ArrwheadRtatin - Rtates the arrwhead randmly every time a new arrw is spawned

17 SundBwClick - Plays the sunds f the bw string being pulled. - The ther scripts in the Lngbw flder handle the lgic fr the lngbw targets ArcheryTarget - This is the script fr a generic archery target. - It invkes a UnityEvent when hit by an arrw. FireSurce - Indicates an bject that can be set n fire. Once n fire this bject can then spread fire when it cmes in cntact with anther FireSurce. ExplsinWbble - Used t make the archery target wbble. Balln BallnClliders BallnHapticBump BallnSpawner - These scripts handle the lgic fr the ballns that spawn when the weeble is hit with an arrw.

18 Samples - There are a few classes that were created specifically t shw sme examples in the sample scene. CntrllerHintsExample - This class shws hw t use the hint system. InteractableExample - This class shws a very simple example f receiving and respnding t messages frm the hand.