Looking to commission a font from someone... please help!

[pinyinviola]

Hi! I am not looking for a typical stylized font here, rather, I need someone to help me create an "expanded" Times New Roman font with some added letters, basically letters with certain diacritics on top. The reason? I am trying to type up some stuff using my own system of Romanized Chinese in various dialects, and only Mandarin is fully supported in fonts like Times New Roman.

Please name this font "Pan-Chinese Pinyin."

Basically, in addition to retaining all the usual stuff in Times New Roman, I need the following letter+diacritic combinations:

A, a, E, e, I, i, O, o, U, u, M, m, N, n, Y, y

each of the above using each of the following diacritics:

macron, vertical macron (short vertical line on top of letter), acute accent, grave accent, caron, circumflex, hook, ring, tilde, umlaut.

Also, I need a "N" that is small sized like "n."

Altogether, that's 161 shapes/characters/glyphs (I'm not familiar with the terminology). Don't forget about the default characters in Times New Roman. Some of these may be already encoded.

It would be nice if there was a way to type it so that for example, press "A" then "~" then you get an A with a tilde, but if that's too hard it's okay.

If FontCreator isn't the right program to create this font, please inform me about other programs that would be able to accomplish this. This font would be a very useful tool for many people as many dialects of Chinese do not have an efficiently designed and systematic way of transcribing sounds into the Roman alphabet.

Thank you for helping!

Pinyinviola

[Bhikkhu Pesala]

Take a look at my Garava Font.

I think you will find that all of those characters are available, except for the letters with vertical macron accent, but a vertical macron combining diacritical mark is included. Small Capital N is included too.

For typing, you should be able to find an existing keyboard for Pinyin, but if not you can use the Microsoft Keyboard Layout Creator (freeware), which i used to create my keyboards for Pali.

Garava is released under GNU license agreement. It can be renamed and modified as long as the license agreement is retained and passed on to anyone who wants to use or modify your Pinyin version.

Times New Roman is a copyrighted font. There are any number of dubious versions of it around. It is not the best font for all publications, as it has been overused.

[pinyinviola]

That is a wonderful font. Are you still fine-tuning it? If you are, would you mind adding characters for Peh-oe-ji, which is a Roman based alphabet for phonetically transcribing Taiwanese/Minnan (a language of Taiwan and Southeastern China). Because m and n can be used as vowels in this language, all tone marks (acute accent, macron, etc.) are available for m and n. Also, Peh-oe-ji uses a unique diacritic, basically a vertical macron.

Basically, the only things preventing me from using Garava for my work is the lack of certain characters still. (Unless I didn't realize that they could be typed out). Sorry, I'm not very experienced with fonts and encoding, I could be just not realizing the characters are included in Garava. If so, please point it out to me.

Thanks!

[Bhikkhu Pesala]

Maybe someone will come along to offer to do the job for you, using Garava, but my advice is, “If you want a job done properly, do it yourself.” You know best how the accents should be placed for Pinyin, and you can keep tweaking it until it is near perfect. All of the hard work is done, all that you need to do is create a few new composites, map them to the Private Use Area, and create a suitable keyboard for input. You can install my Pali extended keyboard in Windows and use that as the basis for your Pinyin keyboard. It supports the typing of vowels with caron, macron, acute, grave, tilde, etc.

The illustration below shows Garava used in Wordpad to type m, n, and y with combining accent vertical macron. A composite would be better, because you could centre the accent, especially over the m.



I don't know Pinyin, I don't do commercial work, and I don't have time for all the correspondence that would be needed. FontCreator is far easier to use than other programs, and for this job the lack of hinting support is not a problem. Garava is already hinted, and you only need to create new composites, not new glyph outlines.

[William]

> Garava is already hinted, and you only need to create new composites, not new glyph outlines.

Does that mean that if someone added a glyph for m with combining accent vertical macron into the Unicode Private Use Area of Garava using just a composite and not altering the contours that the glyph would still be hinted correctly?

> I don't know Pinyin, I don't do commercial work, and I don't have time for all the correspondence that would be needed.

Suppose that one wishes to add a glyph for m with combining accent vertical macron into the Unicode Private Use Area of Garava. Could you possibly say how that would be done once one has chosen a particular Unicode Private Use Area codepoint to use please?

William Overington

15 November 2007

[Bhikkhu Pesala]

Does that mean that if someone added a glyph for m with combining accent vertical macron into the Unicode Private Use Area of Garava using just a composite and not altering the contours that the glyph would still be hinted correctly?

Yes. As far as I know.

Suppose that one wishes to add a glyph for m with combining accent vertical macron into the Unicode Private Use Area of Garava. Could you possibly say how that would be done once one has chosen a particular Unicode Private Use Area codepoint to use please?

Select two or more glyphs in the overview window, open the newly mapped empty glyph and paste them in as a composite. Adjust for position, copy the bearings, and that's it. You can combine simple glyphs with simple glyphs or composites in this way to make a new composite. It is quicker than following the Insert composite glyph member route from the glyph edit window.



The hardest part is deciding on the best mappings to use, and making a usable keyboard to use those PUA mappings. Adding the composites is just a few minutes' work.

[William]

> Select two or more glyphs in the overview window, open the newly mapped empty glyph and paste them in as a composite.

Thank you.

As a test I made a copy of the Garava Regular font and then added U+E201 at the end. The choice was more or less arbitrary, though it is never totally arbitrary in practice when using the Unicode Private Use Area. I wanted a place starting with E, not at the very start of the Unicode Private Use Area and not ending in 0, just in case! So as chose U+E201 as about as general as one can expect to get! I used Format Post... and Generate Names so as to add a Postscript name for the glyph.

I found that one needs to select the m and then hold down the CTRL key and select the uni030D glyph and then use CTRL C to copy.

I then opened the uniE201 glyph window by double clicking from the Overview window.

I then used Edit Paste and was surprised and delighted that the two glyphs went into the new glyph. I did not know that one could do that.

I dragged the macron to the right and at some stage I doubled clicked and the Composite Glyph Properties panel appeared.

After some though I decided to adjust the horizontal position of the macron (which appears to be an offset from its original position rather than an absolute position) to 1280. A figure of 1280 looked reasonable and I thought that a multiple of 256 might work better than some other number as long as the display effect was alright.

> Adjust for position, copy the bearings, and that's it.

As to the "copy the bearings" part I looked at the bearings for the original m glyph and then manually copied the 1569 by right clicking on the composite glyph and then clicking on Properties... and entering the value 1569 into the Advance width box and then clicking OK.

Later I wondered whether I should, or could, have checked the Use this glyphs metrics box on the Composite Glyph Properties panel for the m glyph.

> The hardest part is deciding on the best mappings to use, and making a usable keyboard to use those PUA mappings.

The thing about the mappings is to make a list, on a piece of paper or in a text file, of which of the 161 glyphs are already mapped in regular Unicode as precomposed characters. After that, a list of Private Use Area mappings could be chosen for the remainder.

I don't know how to make the keyboard.

William Overington

15 November 2007

[pinyinviola]

Hmm, I'm very inexperienced with all this font and encoding work, can someone explain the "combining" of a letter with a diacritic? How would I type, for example, an "M" and and "m" with carons on top?

[Bhikkhu Pesala]

I think you are going to need the Microsoft Keyboard Layout Creator at some point. That is the only way to make a keyboard that is usable in applications that don't have the option of customising the keyboard, as Word does, though if you don't mind being restricted to typing Pinyin only in Word, you can customise the keyboard in Word.

There are three ways that European keyboards type accented characters.

1. They assign them to the lowercase or uppercase keys. Very commonly used letters are best here. French uses the 2 key to type é the 7 key to type è and the 9 to type ç.
2. They assign them to AltGr. The standard UK keyboard will type áéíóú with AltGr + Vowel.
3. They assign the accent to a dead key, and combinations with that accent are typed with the usual key after typing the dead key. The French keyboard, for example, uses the [ as a dead key for the circumflex accent. The user type [ followed by a e i o or u to get â ê î ô or û.

For Pinyin, you will probably need to use a lot of dead keys. The choice of dead key is crucial as it interferes with regular typing. For Pali, we need macron accented vowels ā, ī, and ū. If I used the logical choice of hyphen for the dead key, this would make regular typing very difficult as the hyphen is used frequently. Therefore, I used the forward slash (/) instead. I type / followed by a, e, i, o, u to get ā ē ī ō ū or Ā Ē Ī Ō Ū with AEIOU. That is fairly easy to type.

M caron does not have a Unicode code-point. You will have to map it to the Private Use Area or perhaps to Latin Extended D. Then you will need to assign a shortcut such as dead key + followed by m to type it. In Pali, we use both m and n dot above. To type them, I use " followed by m and n (ṁ ṅ). It works OK, but " is used to type quotation marks, so it is a bit awkward. However, I can also use the same method to type äëïöü and ÄËÏÖÜ, so it is a price worth paying if you need lots of accented characters.

Examine my Garava font and Times New Roman in Font Creator to see how composite glyphs like áéíóú are composed from accent ´ plus the vowel. The Wordprocessor or Windows keyboard layout doesn't need to know how a composite glyph is composed, it just needs to know its mapping: á (0225). It doesn't need to know that it composed from a (97) + ´ (0180) — á might be a simple glyph, it will still be typed if the user presses Ctrl + Alter + a or AltGr a.

[William]

> Hmm, I'm very inexperienced with all this font and encoding work, can someone explain the "combining" of a letter with a diacritic?

In a word like café there is an accented letter. It is called e acute. Many languages use accented letters. For historical reasons, some accented letters have a regular Unicode codepoint of their own. The é character has a codepoint of its own. It is a precomposed accented character. However, some characters, such as m macron, do not have a codepoint of their own. There is now a policy of not adding any more precomposed accented characters.

However, accented characters such as m macron and indeed e acute can be expressed as the base letter followed by a combining accent.

When one types an ordinary letter, such as, say, p, all of the "ink" is after where the cursor is and then the cursor moves forward. With a combining diacritic all of the "ink" is before where the cursor is and the cursor does not move. So, the diacritic, often known as "the accent", appears to be on top of the letter just before the cursor. So, one may "construct" an m macron as one proceeds, even though there is not an m macron encoded as a single character in regular Unicode.

However, the font will have one "combining macron" and that needs to be used for both m macron and M macron as well as other macron characters such as n macron and N macron. This can result in ugly typography, as the font designer may have felt the need to have the combining macron glyph high and not too far back, so that N macron would look reasonable. In which case, M macron would have the accent too far to the right to look good, though one could work out the intended meaning alright. The n macron would have the accent look too high, though one could work out the intended meaning alright. The m macron would have the accent look too far to the right and too high, though one could work out the intended meaning alright.

There are broadly two ways around this in order to get good looking typography.

One is to use an Opentype font with an expensive software package such as InDesign. The other is to add precomposed glyphs for M macron, m macron, N macron and n macron to the font, in the Unicode Private Use Area: this has the advantage of making keying the characters easier too, yet does mean that the encoding in a saved file is not regular Unicode.

> How would I type, for example, an "M" and and "m" with carons on top?

I noticed that you mentioned carons. I used macrons in my example. The reason for that is that I have not made any glyphs in the Unicode Private Use Area using carons, but I have made some glyphs in the Unicode Private Use Area using macrons.

http://www.users.globalnet.co.uk/~ngo/QUESTTXT.TTF

http://www.users.globalnet.co.uk/~ngo/CHRONTXT.TTF

If one saves the fonts to hard disc local storage on can then open them in FontCreator and look for the characters using macrons.

They are also linked from the following page.

http://www.users.globalnet.co.uk/~ngo/fonts.htm

I hope that this helps.

William Overington

17 November 2007

[William]

> M caron does not have a Unicode code-point. You will have to map it to the Private Use Area or perhaps to Latin Extended D.

I am wondering why you suggest the mapping of it to Latin Extended D.

Latin Extended D is part of regular Unicode.

If the M caron is mapped to Latin Extended D then a Unicode compliant software package, now or in the future, could reject the character if it is mapped to an unused position in Latin Extended D.

I would advise mapping to the Private Use Area.

If it were me doing the mapping, which it is not, I would proceed as follows.

I would start with the list of required letters.

> A, a, E, e, I, i, O, o, U, u, M, m, N, n, Y, y

I would assign the following part codepoints, using X for a value on which I have yet to decide, using alphabetical order pairs of uppercase and lowercase.

U+EX0. A
U+EX1. a
U+EX2. E
U+EX3. e
U+EX4. I
U+EX5. i
U+EX6. M
U+EX7. m
U+EX8. N
U+EX9. n
U+EXA. O
U+EXB. o
U+EXC. U
U+EXD. u
U+EXE. Y
U+EXF. y

I would then make a copy of the list of required accents.

> macron, vertical macron (short vertical line on top of letter), acute accent, grave accent, caron, circumflex, hook, ring, tilde, umlaut.

I would shuffle those into the order used in the combining diacritics of Unicode, using the table in the following pdf.

http://www.unicode.org/charts/PDF/U0300.pdf

That is grave, acute, circumflex, tilde, macron, umlaut (known as diaeresis), hook, ring, caron, vertical macron (known as combining vertical line above).

I would assign codes as follows.

0 grave
1 acute
2 circumflex
3 tilde
4 macron
5 diaeresis
6 hook
7 ring
8 caron
9 vertical line

I would map the special small N to U+EX8F so that it was with the other characters based around N. However, as Bhikkhu Pesala already has a small capital N at U+E2FE it might be better to use U+E2FE. My reason for saying that is that I am wondering whether mapping the small capital N also to U+EX8F for whatever value of X is chosen might upset the hinting. If it would not upset the hinting I would map also to U+EX8F for whatever value of X is chosen. This would keep the codepoints for your project all together. I am thinking that one day you might write it all up in a paper and a chart with all of the characters together would look good. That chart could show the regular Unicode codepoint for those precomposed characters which have a regular Unicode codepoint.

This would mean that I had, once I had decided on a value for X, 161 Private Use Area codes for use, though if the character were encoded in regular Unicode I would use that and not use the Private Use Area code.

For example, in the above, e acute would be U+EX31, but I would not use that as e acute has the regular Unicode assignment of U+00E9 so I would use U+00E9 for e acute.

http://www.unicode.org/charts/PDF/U0080.pdf

That and other charts are available from the following page.

http://www.unicode.org/charts/

I would then need to decide on a value for X.

I am free to choose any one hexadecimal character for that.

Now, as is well known, I already use various Private Use Area allocations for various things, so, in fact, I would avoid a clash with what I am using for something else. However, in this discussion I shall disregard my own other uses of the Private Use Area and say that I could use any of the 16 hexadecimal characters as I do not wish to influence you in your work on the basis of my experiments with other things.

However, what I would do in relation to this project is have a look at the Code2000 font and find out where there is a gap in the usage of the Private Use Area in that font.

There is no Unicode requirement to do that, but I would do it myself as I would be hoping that there would be a chance that if this project is successful that the maker of the Code2000 font might include my characters in his font at some future time. Clearly there is a better chance of that happening if the codepoints used for this project are not being used for something else in the Code2000 font at the present time.

Before deciding on the value for X, I would also have a look at which Private Use Area codepoints are already in use in the Garava font.

It is kind and generous of Bhikkhu Pesala to invite you to use his Garava font for your project.

I hope that the above notes will be helpful.

William Overington

17 November 2007

[William]

Having looked at the use of the Private Use Area in Garava and in Code2000 and having looked at the Private Use Area mappings mentioned by Bhikkhu Pesala in the viewtopic.php?t=1958 thread, I have come to the conclusion that a good choice for the value of X would be F.

Thus the mappings would be as follows.

U+EF0. A
U+EF1. a
U+EF2. E
U+EF3. e
U+EF4. I
U+EF5. i
U+EF6. M
U+EF7. m
U+EF8. N
U+EF9. n
U+EFA. O
U+EFB. o
U+EFC. U
U+EFD. u
U+EFE. Y
U+EFF. y


...0 grave
...1 acute
...2 circumflex
...3 tilde
...4 macron
...5 diaeresis
...6 hook
...7 ring
...8 caron
...9 vertical line


I would map the special small N to U+EF8F.

For example, in the above, e acute would be U+EF31, but I would not use that as e acute has the regular Unicode assignment of U+00E9 so I would use U+00E9 for e acute.

For example, in the above, m vertical line would be U+EF79.

Of the 161 characters, some are in regular Unicode and some are not.

I know that m macron is not in regular Unicode and, as m is both a character of a width greater than most characters and is a lowercase character, perhaps m macron might be a good example because it is likely to be more difficult than most of the other characters to implement.

Also, please note that to implement the accented characters using i, a dotless i character will be needed. Garava already has a dotless i character so that is not a problem, yet I mention that in case you also try to implement using other fonts.

I suppose that those of the 161 characters which are not in regular Unicode could be added one by one manually into the Private Use Area, but I suspect that a transform script could be written which could then be used to add the required glyphs to any font: there would still need to be some manual adjustment of the positions of the accents, but most of the work would have been done by the Glyph Transformer.

William Overington

19 November 2007