3D Transforms

[tomcrane]

(decisions and suggestions from IIIF/3d#11 (comment))

We need to be able to scale and rotate models to place them in the Scene.

To do this we can add a transforms property to SpecificResource, to specify an ordered sequence of changes to make to the model (or selected part of the model) before placing at the painting annotation target (as above #2253)

From Rob’s draft:

Transformers are instructions as to how to change the resource before rendering in the client, and as such comes between the Selector and the Style in the Annotation rendering workflow.

We need at least two transform classes:

"transforms": [
    {
        "type": "ScaleTransform",
        "x": 1.0,
        "y": 1.1,
        "z": 0.9
    },
    {
        "type": "RotationTransform",
        "x": 0.0,
        "y": 45.0,
        "z": 180.0
    }
]

Note that a full example of a SpecificResource may also have a selector (as #2255) if only part of the model is required.

[edsilv]

Also TranslateTransform ?
You may need to move a set of objects from their original positions in order to arrange them to prevent overlapping for example.

[azaroth42]

Don't need the Translate transform, as that's relative to the Scene, not the model. Thus the target is the right place for the transformation, not the body. Hence we use a selector on the Specific Resource of the target.

[azaroth42]

{
  "id": "https://example.org/iiif/3d/anno1",
  "type": "Annotation",
  "motivation": ["painting"],
  "body": {
    "type": "SpecificResource",
    "source": [
      {
        "id": "pawn-model.glb",
        "type": "Model",
        "format": "application/glb"
      }
    ],
    "transforms": [
      {
        "__comment": "translate to the right place in the model's space",
        "type": "TranslateTransform",
        "x": -15.0,
        "y": -9.0,
        "z": 0.0
      }
    ]
  },
  "target": [
  	{
     "type": "SpecificResource",
     "source": [
      {
        "id": "https://example.org/iiif/scene1/page/p1/1",
        "type": "Scene",
      }
     ],
     "transforms": [
      {
        "__comment": "translate to the right place in the scene's space",      
        "type": "TranslateTransform",
        "x": -100.0,
        "y": -400.0,
        "z": -50.0
      }
     ] 		
	}
  ]
}

[JulieWinchester]

Riffing on the above snippet to add the other 2 transforms at the annotation level, keeping just a translate transform at the target level.

{
  "id": "https://example.org/iiif/3d/anno1",
  "type": "Annotation",
  "motivation": ["painting"],
  "body": {
    "type": "SpecificResource",
    "source": [
      {
        "id": "pawn-model.glb",
        "type": "Model",
        "format": "application/glb"
      }
    ],
    "transforms": [
      {
        "__comment": "Make it half-size",
        "type": "ScaleTransform",
        "x": 0.5,
        "y": 0.5,
        "z": 0.5
      },
      {
        "__comment": "Set it upright",
        "type": "RotationTransform",
        "x": 0.0,
        "y": 0.0,
        "z": -90.0
      },
      {
        "__comment": "translate to the right place in the model's space",
        "type": "TranslateTransform",
        "x": -15.0,
        "y": -9.0,
        "z": 0.0
      }
    ]
  },
  "target": [
  	{
      "type": "SpecificResource",
      "source": [
      {
        "id": "https://example.org/iiif/scene1/page/p1/1",
        "type": "Scene",
      }
      ],
      "transforms": [
      {
        "__comment": "translate to the right place in the scene's space",      
        "type": "TranslateTransform",
        "x": -100.0,
        "y": -400.0,
        "z": -50.0
      }
      ] 		
	  }
  ]
}

[azaroth42]

In thinking this through, I'm -1 on TranslateTransform. The translation doesn't apply to the scene, it applies to the model within the scene, but that's not what the JSON says.

[JulieWinchester]

As was requested during the Basel working meeting on 10/26/2023, adding a note to say that the scale transform as indicated here is using degrees (rather than radians) for reasons of human readability.

[julsraemy]

See https://github.com/IIIF/api/blob/rob_3d/source/presentation/4.0/examples.md where @azaroth42 created a few JSON snippets:

• Most basic case: put the model in the center of the scene
• Positioning: Position the model not at the center of the scene
• Transform the model before putting into the scene (at the center)
• Transform the model and position it in the scene
• Transform the model, including translation, before positioning in the scene

RS from the Google Doc: Transforms are allowed on specific resources, which specify an ordered series of operations to perform on the resource. If there is a Selector, then the transforms operate on the selected part of the resource. There are three Transforms (Scale, Rotate and Translate). They can be repeated in the transforms list.

The result of Selection + Transformation is then annotated to a point in the target Scene with a painting Annotation. The model's coordinate space and the Scene's coordinate space must be aligned, which is done by setting the origin of the model to the origin of the scene. The location within the model's coordinate space is given by a PointSelector to change the alignment to the point selected, rather than the origin.

[julsraemy]

As was requested during the Basel working meeting on 10/26/2023, adding a note to say that the scale transform as indicated here is using degrees (rather than radians) for reasons of human readability.

See #2266

[JulieWinchester]

Though part of this is also stated by @julsraemy above, I wanted to detail some general consensus items from the Basel conference working meeting on October 26, 2023:

• Transforms only pertain to the body of an annotation, a selector like PointSelector only pertains to the annotation target. It is conceivably possible for a selector to pertain to the body, such as in a hypothetical GeomSelector that would select one component primitive from a GLTF scene file, but active support for that is probably not in scope for current efforts.
• Transforms in most cases act independent from the Scene and should be applied prior to the annotation being placed within the Scene. In this sense, most transforms are "pre-Scene." Target selectors like PointSelector, meanwhile, explicitly reference the Scene.
• There are at least three allowed transforms for model annotations: Scale, Rotate, and Translate. Consensus was reached in favor of the translate transform. The transforms can be repeated in the transforms list, and they are applied in the order they are specified (e.g., there is no a priori ordering of application of different transform types).

[tomcrane]

Consensus was reached in favor of the translate transform.

Some further details from @JulieWinchester to my question:

If you do have a TranslateTransform on the body, it applies within that body's space - the model's space.
But what can you achieve like that that you can't achieve simply by positioning the anno with a point selector, after you've done whatever scale and rotate transforms you need? TranslateTransform is not actually needed because positioning does the same thing.

I think the issue is two-fold here. One (less practical) is that 3D specialists tend to think in terms of the three basic transforms as a set, and if we only implement 2/3, we’re going to spend all of our time answering the question of why we don’t have the third. Two (more practical) is that it significantly simplifies situations where you might want to “correct” a particular model that has been placed off-origin. When we worked through simple examples like placing an off-origin pawn multiple times to have multiple pawns on a 3D model chessboard, and having the entire game placed in 3D space, being able to have TranslateTransform as well as a point selector really simplified things.

[azaroth42]

Proposal from @JulieWinchester that negative numbers are allowed for flipping axes. E.g. to go from right to left handed.
Cannot be achieved with rotation.

(+1 from me)

[JulieWinchester]

In addition to the three standard transforms TranslateTransform, RotateTransform, and ScaleTransform, I think it would make sense to provide an additional convenience transform TranslateToOriginTransform (or some other name similar to that). TranslateToOriginTransform would translate the model such that the model centroid (where centroid is defined in a manner left to the implementation) would be aligned to the model's local coordinate origin, basically "centering" the model in its own local coordinate space.

{
    "type": "SpecificResource",
    "source": {
        "id": "https://example.org/iiif/assets/model1.glb",
        "type": "Model"
    },
    "transform": [
      {
        "type": "TranslateToOriginTransform",    
      }
    ]
}

This kind of operation - centering a model before carrying out further transforms like scaling or rotating - is extremely common when working with 3D. It's also not always immediately obvious from an end-user perspective what is the off-set of the model centroid from the origin, unless one delves directly into the 3D coordinate data. Because of this, if a user wanted to use TranslateTransform to accomplish this task, it may not be trivially easy to know what the correct translation off-set would be to carry out this translation. Despite this, on the implementation end, TranslateToOriginTransform is a very simple convenience/gloss on top of TranslateTransform, since the implementation almost certainly already has access to the kinds of calculations needed, and so the implementation should be able to handle the math for the end user without much difficulty.