Complete Composites

[Leon_Gauthier]

I looked but did not find an appropriate topic I could add to so I created this one.

I just want to say how pleased I am with the Complete Composites function. It is exactly what I wanted most from FC 9.0, the ability to add Sanskrit transliteration characters to an existing font. And this function made the process incredibly simple, easy and accurate. Yes, it was even fun!

Not that I am any sort of an expert but, I also wanted to share what I think helped to made this so easy and pleasant:
I had compiled a list of the 38 characters I needed along with their hexadecimal Unicode values in a document for reference.

I read the tutorial from beginning to end and although it raised a few questions in my mind, they cleared up as I examined the CompositeData.xml file. Although it was a little vexing to discover that the file used decimal values for the glyphs, I was able to confirm that the correct letters and accents were going to be composited. I am still not sure why the accents would be in the private use area but it seemed clear that if the accents were not there (which they weren’t in my case), the ones at their normal Unicode address would be used.

In my case, the accents were not present so I added them first with Insert Character, using their hexadecimal addresses copied and pasted from my reference document. Although, no composite characters existed in the font, the little tilde, the dot above and the acute glyphs all appeared completed. I copied the existing macron @ $00AF to the new, empty, Modifier Letter Macron @ $02C9 and copied the new Dot Above to the new Combining Dot Below, repositioning it.

With the accents in place, I fed my list of codepoints into Insert Characters one group at a time out of caution. After entering an ordered list of the desired characters (like those with Dot Above), the new glyphs are created and selected. A right-click followed by a click on Complete Composites and voilà, we have newly completed composites.

Next, I get to see how well this all works for the italic, bold and bolditalic fonts.

Thanks guys for a great program. And thanks to Bhikkhu for maintaining the massive CompositeData.xml file (48,041 lines) which makes this possible.

[Bhikkhu_Pesala]

The decimal code-points in CompositeData.xml can be written in the hexadecimal form $25 instead of decimal 37, etc. For example, the definition for A Grave in decimal:

<Composite><!-- A Grave -->
<GlyphMapping>192</GlyphMapping>
	<Member id="1">
	<GlyphMapping>65</GlyphMapping>
		<UseMetrics>TRUE</UseMetrics>
	</Member>
	<Member id="2">
	<GlyphMapping>57365</GlyphMapping>
		<GlyphMapping>96</GlyphMapping>
		<Pos>Auto</Pos>
	</Member>
	</Composite>

Would be written like this in hexadecimal:

<Composite><!-- A Grave -->
<GlyphMapping>$C0</GlyphMapping>
	<Member id="1">
	<GlyphMapping>$41</GlyphMapping>
		<UseMetrics>TRUE</UseMetrics>
	</Member>
	<Member id="2">
	<GlyphMapping>$E015</GlyphMapping>
		<GlyphMapping>$60</GlyphMapping>
		<Pos>Auto</Pos>
	</Member>
	</Composite>

The coding methods can be mixed in the file too, so it might be a useful way to distinguish your changes from the original definitions.

Perhaps in the future, CompositeData.xml will be able to use glyphnames instead of code-points, then we can get away from using large parts of the PUA for composing Stacking Diactrics, Petite Capitals, OldStyle Figures, etc.

The code-points in the PUA are for low-profile diacritics. These are useful for some fonts to apply to uppercase base glyphs. If they do not exist, the fall-back glyphs will be used instead, e.g. if $E015 (low profile grave) does not exist, $60 (grave accent) will be used.

Normally, one would use the code-points in the Spacing Modifier Letter or Combining Diacritical Marks character sets.

The PUA is also used for Stacking Diacritics needed for Vietnamese.

Stacking diacritics, such as those used to write Vietnamese, pose difficult problems for the font designer. For example, U+1EA4 : LATIN CAPITAL LETTER A WITH CIRCUMFLEX AND ACUTE (decimal 7844) requires too much vertical space if the standard diacritics are just stacked vertically. Some designers scale down the diacritics vertically and/or move them closer together than normal to save space. Other designers may opt to place them side by side. I found it is best to define the stacking diacritics in the Private Use Area. CompositeData.xml uses the range from decimal 57344-57364. After inserting these glyphs with “Insert Characters” use Complete Composites to compose them roughly from the single accents. Then decompose them and adjust them to fit the uppercase vowel E.

[Bhikkhu_Pesala]

@Leon Gauthier

I copied the existing macron @ $00AF to the new, empty, Modifier Letter Macron @ $02C9 and copied the new Dot Above to the new Combining Dot Below, repositioning it.

Note that modifier letter macron can be composed from macron using Complete Composites, and macron can be composed from the hyphen.

Ccombining dot below will be composed from periodcentre, and periodcentre will be composed from period (it is scaled down by 85%). I try to avoid using nested composites, ∴ after composting periodcentre, I will make it simple ∵ I know it is used by many other composite glyphs.

CompositeData.xml is designed to make the most of any existing glyph contours that might exist in a font. I assume that someone creating a font from scratch will create a hyphen, but may not necessarily create a contour for the macron. Likewise, they will probably create a plus sign, but probably not a multiply, or minus sign, so both of the latter glyphs can use complete composites to create a contour from the plus sign. Obviously, some editing will be needed to rotate the multiply by 45° and to remove the vertical stroke from the minus sign, but using complete composites ensure consistent advance widths and stroke weights.

It is important to realise that the feature should sometimes only be used as a design aid, and should not be applied to simple glyphs that have already been completed, e.g. to percent or per mille.

However, most composite glyphs are designed to be left as composites: e.g. àèìòù etc., then if you later redesign the base glyph or the accent, the composites will update to match.

[Bhikkhu_Pesala]

@Leon Gauthier

With the accents in place, I fed my list of codepoints into Insert Characters one group at a time out of caution. After entering an ordered list of the desired characters (like those with Dot Above), the new glyphs are created and selected. A right-click followed by a click on Complete Composites and voilà, we have newly completed composites.

The Glyph Transform feature can do both at once: insert a range of characters, then compose them.

Take a look at the Eastern Europe or Vietnamese scripts to see how they work, then design your own Sanskrit transform script and save it for reuse.

[Leon_Gauthier]

Well, For the italic font, my little trick of copying the existing macron @ $00AF to the empty Modifier Letter Macron @ $02C9 did not work as well as copying the new Dot Above to the Combining Dot Below. I adjusted the copied dot down 617 and left 225 to put the pair on the 20 angle of the italic. The dots seem to look good to me:

However, the copied macron seems to be all over the place in the combinations and A single adjustment is not likely to fix it:

Any Ideas about what I am doing wrong here?
Perhaps I just have to manually adjust each of these.

Bhikkhu, I will have to digest your post a bit.

[Leon_Gauthier]

Bhikkhu Pesala said
However, most composite glyphs are designed to be left as composites: e.g. àèìòù etc., then if you later redesign the base glyph or the accent, the composites will update to match.

Also, aren’t the files smaller with composites?

[Leon_Gauthier]

Duh, manual adjustment took a minute and I even made the macron a little longer.

[Leon_Gauthier]

Note that modifier letter macron can be composed from macron using Complete Composites, and macron can be composed from the hyphen.

Aren’t these composites controlled by CompositeData.xml? I can’t seem to find “modifier letter macron” or “Modifier Letter Macron” in the file where I’d expect to see it in a comment like:

Take a look at the Eastern Europe or Vietnamese scripts to see how they work, then design your own Sanskrit transform script and save it for reuse.

Where dat?

[Bhikkhu_Pesala]

@Leon Gauthier

Any Ideas about what I am doing wrong here?

After adjusting the diacritics, make the composites simple, then complete composites again. Note, as I said above, that the low profile diacritics are used for Uppercase, while the regular diacritics are used for lowercase. You may not need the low profile diacritics for your font.

@Leon Gauthier

Also, aren’t the files smaller with composites?

Yes. There’s a worthwhile saving in file sizes if using composites.

@Leon Gauthier

Note that modifier letter macron can be composed from macron using Complete Composites, and macron can be composed from the hyphen.

Aren’t these composites controlled by CompositeData.xml? I can’t seem to find “modifier letter macron” or “Modifier Letter Macron” in the file where I’d expect to see it in a comment like:

<Composite><!-- Macron -->
<GlyphMapping>713</GlyphMapping>

@Leon Gauthier

Take a look at the Eastern Europe or Vietnamese scripts to see how they work, then design your own Sanskrit transform script and save it for reuse.

Where dat?

From the Tools menu, Glyph Transform, click the Open folder icon to find the predefined transform scripts.

[Leon_Gauthier]

FYI: Glyph Transform does not open unless there is a project or font loaded.
Thanks, I’ll have a look.

I would not care to count the number of "macron"s that are in that file … How many times might “Modifier Letter Macron” be found?

I would not have expected that single glyphs are “completed composites” but this is good to know … all that copying I was doing was unnecessary.

[Leon_Gauthier]

Codepoints, Glyph Name, Glyph Index, Unicode Name, period, full stop, middle dot, periodcentered, bullet operator hex this, decimal that …

You guys just make this stuff up in order to keep ordinary humans like me out of your game!!!

Combining dot below will be composed from periodcentre, and periodcentre will be composed from period (it is scaled down by 85%).

I search for “periodcentere” in TNR font and find “periodcentered” @ $2219, #1020, also called “Bullet Operator”. I search in CompositeData.xml for “1020” to see how you composed periodcentre from the period …

_____1020
_____
_________961
_________TRUE
_____
…

Oops, that is not it. Oh yeah, I want decimal($2219) = 8729. Search again …

8729
_____
_____183
_________TRUE
_____
_____

Yup, that’s it. I search for #183 in TNS just to confirm that it is, indeed, the period and what do I find? “ø” which is called LATIN SMALL LETTER O WITH STROKE or oslash. At least it is not Rho Stroke!

If I had any hair, I’d be pulling it out by now!

Maybe, I need to look for Hex(183) = $B7 … What do I find? “·” the middle dot!!!

Well, from my own vast experience, I know that the period is somewhere in the ASCII range so I just browse a bit and find it @ $2E, #17, also known as FULL STOP which I am going to do now!

(BTW: FWIW, in the TNR, the period and the periodcentered are the same size.)

[Leon_Gauthier]

Wow, that Vietnamese.xml script is compact for the task it performs. (you have already done the heavy lifting with CompositeData.xml)

I have a couple of questions:

1. Can you insert s in long comments between in order to keep them on the screen?
2. I assume you cannot insert s in the list. But, could you break the lists into several smaller lists or would the first terminate the input?
3. What does the True/False accomplish?

Actually, I have a very short list of accents and characters so it is only the comments which might need shortening.

Also, it is important to keep these lists in ascending order, isn’t it?

Thanks, Leon

[Bhikkhu_Pesala]

@Leon Gauthier

(BTW: FWIW, in the TNR, the period and the periodcentered are the same size.)

That’s the Bullet Operator. The periodcentred.1 (middle-dot) decimal 183 is smaller. 210 funits diameter as opposed to 222 funits.

Since type foundries don’t use consistent glyph names, it would be better to stick to searching for decimal code-points. Use the Ctrl J shortcut to switch to Decimal codepoint display, and Ctrl K to switch back to hex.

[Bhikkhu_Pesala]

1. If you’re editing the Transform scripts in Notepad, enable word wrap from the Format menu. I use Notepad2, which is more powerful as a text editor, without being over complex. You might find it easier to edit the scripts in the Transform Wizard dialogue.
2. See 1. You can, of course, use insert characters in several stages to keep the list shorter, but see 3.
3. In the Insert Characters script in the Transform Wizard, there’s a checkbox for applying the subsequent commands to the characters just inserted, i.e. to keep them selected and complete composites after inserting them. If you look at the OldStyle Figures script, you can see how I insert the characters and process them in stages to get the desired result, and keep them in numerical order 0-9.
   
   
   

Also, it is important to keep these lists in ascending order, isn’t it?

Yes. You can insert a range with 58051-58053, or separate single codepoints with a comma: 58055, 58057.

The Vietnamese script inserts the diacritics first so that they are available for composing composites later. Composites are decomposed if they are going to be used as composite glyph members later, to avoid nested composites.

[Leon_Gauthier]

Oh, man!!! My new script seems to have worked! SIX CLICKS and we are done!