How to use ClothGrabComponent?

Hi @jamesic,

ClothGrabComponent isn't a component you attach directly to your cloth entity to make it grab-able; instead, it's a component you attach to a separate "grabber" entity which you can then use to grab and manipulate the cloth.

For example, you can create a cloth grabber entity with a ClothGrabComponent like so:

let clothGrabber = Entity(components: [ModelComponent(mesh: .generateSphere(radius: 0.1), materials: [SimpleMaterial()]),
                                       OpacityComponent(opacity: 0.5),
                                       InputTargetComponent(),
                                       CollisionComponent(shapes: [.generateSphere(radius: 0.1)]),
                                       ClothGrabComponent(mode: .volume(shape: .sphere(.init(radius: 0.1))))])

Here, I've set the cloth grab component's mode to volume(shape:) so that it grabs cloth within a 0.1m radius sphere of its position, but you could use the ray mode to grab cloth the grabber entity is pointing toward instead. I've also attached a ModelComponent to make the grabber visible in the scene, and an InputTargetComponent and a CollisionComponent to make it targetable by gestures.

Next, you can write a gesture that makes this entity respond to drag inputs:

@State var isDragging = false
@State var dragStartPosition: SIMD3<Float> = .zero

// ...

.gesture(
    // Make the cloth grabber draggable.
    DragGesture(minimumDistance: 0)
        .targetedToEntity(clothGrabber)
        .onChanged { event in
            if !isDragging {
                isDragging = true
                dragStartPosition = clothGrabber.position
                // Start grabbing the cloth when the drag gesture begins.
                clothGrabber.components[ClothGrabComponent.self]?.isGrabbing = true
            }
            // Move the grabber with the drag translation.
            let translation = event.convert(event.translation3D, from: .local, to: .scene)
            clothGrabber.position = dragStartPosition + translation
        }.onEnded { event in
            isDragging = false
            // Stop grabbing the cloth when the drag gesture ends.
            clothGrabber.components[ClothGrabComponent.self]?.isGrabbing = false
        }
)

By setting the cloth grab component's isGrabbing property to true when the drag gesture begins, and to false when the drag gesture ends, you can now grab the cloth with the grabber by initiating a drag when it overlaps the cloth, and release the cloth by ending the drag.

To achieve more natural interactions, consider attaching the InputTargetComponent and CollisionComponent to the cloth entity itself (so that it can be the target of the drag gesture instead of the grabber) and positioning the cloth grabber entity at the hit location of the drag gesture on the cloth. One caveat to this approach is that you'll need to use a collision shape that closely conforms to the shape of your cloth, which can be difficult given that the cloth itself is deformable. Depending on your use case, you may find that making a static surface your cloth is in close-contact with the gesture target yields even better results. For example, if your cloth is simulating clothing affixed to a static character model, consider targeting your gestures to the character model itself and using those interaction locations to drive the grab component.

Let me know if you have any questions!

Hi @jamesic,

I wanted to follow up with another approach you could take—particularly if you're building for macOS—that utilizes cloth grab component's ray mode to grab cloth anywhere in the scene (without requiring you to set up input targets and colliders for your cloth bodies):

struct ClothGrabView: View {
    // The current location of the pointer.
    @State var pointerLocation: CGPoint?
    // Whether the pointer is pressed.
    @State var isPointerPressed = false

    // The cloth grabber entity, set to ray mode so that it grabs cloth along a ray in the direction it's facing.
    let clothGrabber = Entity(components: [ClothGrabComponent(mode: .ray)])

    var body: some View {
        RealityView { content in
            // Set up the cloth simulation and add the cloth grabber to the scene...
            
            // Subscribe to `ClothSimulationEvents.BeforeUpdate` to receive a callback before the cloth simulation updates.
            _ = content.subscribe(to: ClothSimulationEvents.BeforeUpdate.self) { [content] event in
                // Get the ray from the camera through the current pointer location.
                guard let location = pointerLocation,
                      let ray = content.ray(through: location, in: .local, to: .scene) else {
                    return
                }
                
                // Position and orient the grabber to match the ray.
                clothGrabber.setPosition(ray.origin, relativeTo: nil)
                clothGrabber.setOrientation(simd_quatf(from: [0, 0, -1], to: ray.direction), relativeTo: nil)

                // Initiate a grab while the pointer is pressed.
                clothGrabber.components[ClothGrabComponent.self]?.isGrabbing = isPointerPressed
            }
        }
        // Track the pointer's location.
        .onContinuousHover(coordinateSpace: .local) { phase in
            guard case .active(let location) = phase else {
                pointerLocation = nil
                return
            }

            pointerLocation = location
        }
        // Track the pointer's location and state when it's pressed.
        .gesture(
            DragGesture(minimumDistance: 0, coordinateSpace: .local)
                .onChanged { value in
                    pointerLocation = value.location
                    isPointerPressed = true
                }
                .onEnded { _ in isPointerPressed = false }
        )
    }
}

This approach works by constantly tracking the mouse pointer's position, computing a ray from the camera through that position, and then orienting the cloth grabber such that it faces along that ray. Once the cloth grab component's isGrabbing property is set to true it will then grab any vertices the ray hits.

Let me know if you have any questions.