TLDR: NSLayoutManager's textContainer(forGlyphAt:effectiveRange:) and lineFragmentRect(forGlyphRange:effectiveRange:) are returning inconsistent results.
Context: I'm developing a word processing app that paginates from an NSTextStorage using NSLayoutManager. My app uses a text attribute (.columnType) to paginate sub-ranges of the text at a time, ensuring that each columnRange gets a container (or series of containers across page breaks) to fit. This is to support both multi-column and standard full-page-width content.
After any user edit, I update pagination data in my Paginator model class. I calcuate frames/sizes for the views/containers, along with what superview they belong to (page). The UI updates accordingly.
In order to determine whether the columnRange has overflowed from a container due to a page break OR whether the range of text hasn't overflowed its container and is actually using less space than available and should be sized down, I call both:
layoutManager.textContainer(forGlyphAt: lastGlyphOfColumn, effectiveRange: &actualGlyphRangeInContainer)`
// and
`layoutManager.lineFragmentRect(forGlyphAt: lastGlyphOfColumn, effectiveRange: nil)
Apple Documentation notes that both these calls force glyph generation and layout. As I'm in early development, I have not set non-contiguous layout. So these should be causing full layout, assuring accurate return values.
Or so I'd hoped.
This does work fine in many cases. I edit. Pagination works. But then I'll encounter UI-breaking inconsistent returns from these two calls. By inconsistent, I mean that the second call returns a line fragment rect that is in the container coordinates of A DIFFERENT container than the container returned by the first call. To be specific, the line fragment rect seems to be in the coordinates of the container that comes next in layoutManager.textContainers.
Example Code:
if !layoutManager.textContainers.indices.contains(i) {
containerToUse = createTextContainer(with: availableSize)
layoutManager.addTextContainer(containerToUse)
} else {
// We have a container already but it may be
// the wrong size.
containerToUse = layoutManager.textContainers[i]
if containerToUse.size.width != availableSize.width {
// Mandatory that we resize if we don't have
// a matching width. Height resizing is not
// mandatory and requires a layout check below.
containerToUse.size = availableSize
}
}
let glyphRange = layoutManager.glyphRange(forCharacterRange: remainingColumnRange, actualCharacterRange: nil)
let lastGlyphOfColumn = NSMaxRange(glyphRange) - 1
var containerForLastGlyphOfColumn = layoutManager.textContainer(forGlyphAt: lastGlyphOfColumn, effectiveRange: &actualGlyphRangeInContainer)
if containerForLastGlyphOfColumn != containerToUse
&& containerToUse.size.height < availableSize.height {
// If we are here, we overflowed the container,
// BUT the container we overflowed didn't use
// the maximum remaining page space (this
// means it was a pre-existing container that
// needs to be sized up and checked once more).
// NOTE RE: THE BUG:
// at this point, prints show...
// containerToUse.size.height
// =628
// availableSize.height
// =648
containerToUse.size = availableSize
containerForLastGlyphOfColumn = layoutManager.textContainer(forGlyphAt: lastGlyphOfColumn, effectiveRange: &actualGlyphRangeInContainer)
}
// We now check again, knowing that the container we
// are testing flow into is the max size it can be.
if containerForLastGlyphOfColumn != containerToUse {
// If we are here, we have overflowed the
// container, so containerToUse size SHOULD be
// final/accurate, since it is fully used.
actualCharRangeInContainer = layoutManager.characterRange(forGlyphRange: actualGlyphRangeInContainer, actualGlyphRange: nil)
// Start of overflow range is the first character
// in the container that was overflowed into.
let overflowLoc = actualCharRangeInContainer.location
remainingColumnRange = NSRange(location: overflowLoc, length: remainingColumnRange.length - overflowLoc)
// Update page count as we have broken to a new page
currentPage += 1
} else {
// If we are here, we have NOT overflowed
// from the container. BUT...
// THE BUG:
// ***** HERE IS THE BUG! *****
lineFragmentRectForLastChar = layoutManager.lineFragmentRect(forGlyphAt: lastGlyphOfColumn, effectiveRange: nil)
let usedHeight = lineFragmentRectForLastChar.maxY
// BUG: ^The lines of code above return a
// fragment rect that is in the coordinates
// of the WRONG text container. Prints show:
// usedHeight
// =14
// usedHeight shouldn't be just 14 if this is
// the SAME container that, when it was 628
// high, resulted in text overflowing.
// Therefore, the line fragment here seems
// to be in the coordinates of the ENSUING
// container that we overflowed INTO, but
// that shouldn't be possible, since we're in
// a closure for which we know:
//
// containerForLastGlyphOfColumn == containerToUse
//
// If the last glyph container is the container
// we just had to size UP, why does the final
// glyph line fragment rect have a maxY of 14!?
// Including ensuing code below only for context.
if usedHeight < containerToUse.size.height {
// Adjust container size down to usedRect
containerToUse.size = CGSize(width: containerToUse.size.width, height: usedHeight)
} else if usedHeight == availableSize.height {
// We didn't force break to a new page BUT
// we've used exactly the height of our page
// to layout this column range, so need to
// break to a new page for any ensuing text
// columns.
currentPage += 1
} else if usedHeight > containerToUse.size.height {
// We should have caught this earlier. Text
// has overflowed, but this should've been
// caught when we checked
// containerForLastGlyphOfColumn !=
// containerToUse.
//
// Note: this error has never thrown.
throw PaginationError.unknownError("Oops.")
}
}
Per my comments in the code block above, I don't understand why the very same text container that just overflowed and so had to be sized up from 628 to 648 in order to try to fit a glyph would now report that same glyph as both being IN that same container and having a line fragment rect with a maxY of just 14. A glyph couldn't fit in a container when it was 628 high, but if I size it up to 648, it only needs 14?
There's something very weird going on here. Working with NSLayoutManager is a bit of a nightmare given the unclear documentation.
Any help or insight here would be massively, massively appreciated.
