refresh of selected font folder is not possible

Get help with MainType here. Please do not post feature requests or bug reports here.
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

refresh of selected font folder is not possible

Post by thibaud »

My main font library is quite large (around 100k fonts)
it's added via a single "keep in sync" folder set to manual refresh mode.
whenever I refresh the whole folder via CTRL-R or individual subfolders via either the button "synchronize the selected font folder" in the group pane or the contextual menu of the subfolder.
the library refresh takes a very long time: 7 minutes :shock:

Considering that the library is stored on a quite fast local raid5 array (8 disks) are these times considered normal ?

I assumed refreshing a single subfolder holding a few fonts would take considerably less time, but it isn't the case (it seems like the whole library is updated each time) is this normal ?

What can I do to improve this refresh performance ?
Last edited by thibaud on Sat Jan 27, 2018 11:19 am, edited 1 time in total.
Erwin Denissen
Moderator
Moderator
Posts: 11108
Joined: Fri Oct 04, 2002 12:41 am
Location: Bilthoven, The Netherlands
Contact:

Re: refresh performance of a large font folder

Post by Erwin Denissen »

The sync feature will be applied to the whole folder and during that process it has to read all font resources to ensure they aren't changed. 7 minutes sounds pretty fast to me.
Erwin Denissen
High-Logic
Proven Font Technology
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh performance of a large font folder

Post by thibaud »

ok, but isn't there room for improvement here ?
most library manager (be it music or pictures) even when dealing with large libraries don't impose a 7 minutes delay every time you import new items.
Bhikkhu Pesala
Top Typographer
Top Typographer
Posts: 9873
Joined: Tue Oct 29, 2002 5:28 am
Location: Seven Kings, London UK
Contact:

Re: refresh performance of a large font folder

Post by Bhikkhu Pesala »

thibaud wrote:My main font library is quite large (around 100k fonts)
the library refresh takes a very long time: 7 minutes :shock:

I assumed refreshing a single subfolder holding a few fonts would take considerably less time, but it isn't the case (it seems like the whole library is updated each time) is this normal ?

What can I do to improve this refresh performance ?
My maths tells me that 420 seconds to examine 100,000 font files isn't too bad. My library is only about 4K of fonts, and refreshing the fonts folders takes less than 10 seconds on a standard PC.

I have my fonts organised into subfolders on my hard drive, and MainType creates groups from these folders. Most folders never change. The folders that do change regularly, like my OpenType folder for fonts generated by FontCreator are refreshed, the other are not. The Fonts Catalogue folder doesn't change often, but it makes it easier to update my online catalogue if it's included in my Font Folders.

I think you should organise your font collection so that MainType only has to refresh those folders that are likely to change.
Font Folders.png
Font Folders.png (21.3 KiB) Viewed 17381 times
My FontsReviews: MainTypeFont CreatorHelpFC15 + MT12.0 @ Win 10 64-bit build 19045.2486
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh performance of a large font folder

Post by thibaud »

Bhikkhu Pesala wrote: I think you should organise your font collection so that MainType only has to refresh those folders that are likely to change.
My font library is tightly organized, it consist of OTF files only, structured like so:

Code: Select all

+ FONT LIBRARY
|   + FoundryName
|   |   + FamilyName
|   |   |    FontName-bold.otf
|   |   |    FontName-light.otf
|   |   |    FontName-regular.otf
|   |   |    ...
|   |   - FamilyName
|   |   - ...
|   - FoundryName
|   - ...
Whenever I add a new font family it naturally goes into this structure.
Changing the way I store and manage incoming fonts to work around how the refresh works, (like having a separate incoming structure that gets flushed and integrated into the main structure periodically) doesn't sound like a practical solution (extra work, delayed duplicate management, etc..)

Just to clarify (I noticed the subject line is missleading) I'm not suggesting there are performance issues in the refresh routine itself (I suppose it's indeed quite fast for what it does- though I still have to make some test because I suspect something fishy here as the scaling isn't linear 4k/10sec & 100k/420sec :? ) but rather questioning the way the refresh works by forcing an entire re-scan of the library.
Erwin Denissen wrote:The sync feature will be applied to the whole folder and during that process it has to read all font resources to ensure they aren't changed. 7 minutes sounds pretty fast to me.
Why is this so ?
Are there any technical reason/limitation why the refresh is not operated on the selected folder only ?
Having a action that says "synchronize the selected font folder" update the whole library instead of the selected folder looks like a bug from a user perspective.

Would you consider a more optimized solution that would rely on some sort of directory change notification from the filesystem for a future Maintype version perhaps ?
Erwin Denissen
Moderator
Moderator
Posts: 11108
Joined: Fri Oct 04, 2002 12:41 am
Location: Bilthoven, The Netherlands
Contact:

Re: refresh performance of a large font folder

Post by Erwin Denissen »

thibaud wrote:
Erwin Denissen wrote:The sync feature will be applied to the whole folder and during that process it has to read all font resources to ensure they aren't changed. 7 minutes sounds pretty fast to me.
Why is this so ?
Are there any technical reason/limitation why the refresh is not operated on the selected folder only ?
Having a action that says "synchronize the selected font folder" update the whole library instead of the selected folder looks like a bug from a user perspective.
I understand, so I'll add this to our to-do list.
thibaud wrote:Would you consider a more optimized solution that would rely on some sort of directory change notification from the filesystem for a future Maintype version perhaps ?
We initially spend months working on this approach, but came to the sad conclusion the file system change notifications are not always guaranteed to get fired, so we decided take another approach.
Erwin Denissen
High-Logic
Proven Font Technology
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh performance of a large font folder

Post by thibaud »

Thanks a lot for the prompt answers.
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh performance of a large font folder

Post by thibaud »

I just purchased and installed the new (MainType 8.0) version.

Unfortunately things didn't improve regarding this issue with folder refreshing (quite the contrary).

Whenever I want to add a font to my 120k fonts library I have to wait about 30 minutes for MainType to finish it's refresh process of the whole library. (note that MainType is also barely responsive during that process)

I don't know how other user manage their library, but this is definitively a show-stopper for me.

As I explained earlier the tooltip on the group panel's sync button that says: "Synchronise the selected font folder with your file system,"

Image

Is exactly what I'd like to be able to do: syncing a single folder within my library structure.

Can I expect a solution regarding this issue at some point or should I start looking elsewhere?

Thanks a lot.

PS: perhaps file system change notifications did improve...
Perhaps alternative methods of watching a folder structure exists..
for what it's worth I'm using a multi-platform music player https://roonlabs.com/ that is remarkably prompt at updating the library (and a way larger library than my font library) whenever files are added to it's folder structure
Last edited by Bhikkhu Pesala on Fri Jan 26, 2018 6:34 pm, edited 1 time in total.
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh of selected font folder is not possible

Post by thibaud »

Hello,
Are there any improvement planned for the refresh/folder sync ?
Erwin Denissen
Moderator
Moderator
Posts: 11108
Joined: Fri Oct 04, 2002 12:41 am
Location: Bilthoven, The Netherlands
Contact:

Re: refresh of selected font folder is not possible

Post by Erwin Denissen »

We'll certainly look into solutions for this issue, so hopefully the next version of MainType will become faster in this respect.
Erwin Denissen
High-Logic
Proven Font Technology
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh of selected font folder is not possible

Post by thibaud »

great news thanks for the update Erwin.
for what it's worth none of the alternative I tried came close to maintype overall
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh of selected font folder is not possible

Post by thibaud »

I purchased the upgrade and installed Maintype 9.
Unfortunately the contextual menu entry "Refresh Folder" when selecting a specific folder in the groups Pane Still does not refresh the selected folder, but the entire Library... and it still takes ages.
Erwin Denissen
Moderator
Moderator
Posts: 11108
Joined: Fri Oct 04, 2002 12:41 am
Location: Bilthoven, The Netherlands
Contact:

Re: refresh of selected font folder is not possible

Post by Erwin Denissen »

I'm sorry, but it didn't make it into version 9. It requires some heavy refactoring, and it seems to be more complicated than we expected.
Erwin Denissen
High-Logic
Proven Font Technology
thibaud
Posts: 75
Joined: Fri Jan 27, 2006 8:20 pm
Contact:

Re: refresh of selected font folder is not possible

Post by thibaud »

before purchasing the maintype 10 upgrade... I'd like to know, was this issue addressed in version 10 ?
Erwin Denissen
Moderator
Moderator
Posts: 11108
Joined: Fri Oct 04, 2002 12:41 am
Location: Bilthoven, The Netherlands
Contact:

Re: refresh of selected font folder is not possible

Post by Erwin Denissen »

No, not yet. It is high on the priority list.
Erwin Denissen
High-Logic
Proven Font Technology
Post Reply