The pros & cons and ups & downs of OCR and Scrivener

The pros & cons and ups & downs of OCR and Scrivener

(Warning: tech talk ahead!)

I’ve been putting research documents into Scrivener, assuming that they were searchable. After all, one oft-stated advantage of using Scrivener is that you have all your documents in one place.

Dissertating with Scrivener « The Junto

It’s true that I can put everything and anything into Scrivener, but I also need to be able to search within those documents. I mistakenly assumed that one of Scrivener’s many superpowers was the ability to make all documents searchable. In other words, I assumed that Scrivener utilized OCR (Optical Character Recognition). Not so. :-(

Having searchable documents is important for my current WIP because it’s set in the 16th century, and a number of the resources are rare and/or ancient and only available on BooksGoogle or InternetArchive. I’ve taken to clipping relevant parts of such documents (shift-control-4 on a Mac) or exporting them whole as PDFs before sending them to Scrivener. The clips are a type of image, so they need OCR to be searched, and most PDFs are not searchable as well.

And so I began to look at ways to make documents searchable before putting them in Scrivener. In the process, I discovered that anything to do with OCR opened a bottomless pit. I will try to keep this simple.

Dedicated OCR software

One possibility would be to invest in a software programme dedicated to making documents OCR searchable. The highest-rated programme for Mac is ABBYY FineReader Pro, available on trial for 30 days. I tested it out on a clip (below), and in seconds had a searchable Word document that beautifully preserved the formatting of the original.

This is the original clip:

And this is the searchable Word document:

Wow.

Databases that make documents searchable

The other possibility would be to use a database that automatically makes documents OCR searchable. The advantage of using such a database is that it is—duh—a database, a logical place to store research documents. … which brings me to OneNote and EverNote.

evernote logo design | Flickr - Photo Sharing! Best 7 Evernote Alternatives: Try These Great Apps Like ...

Both EverNote and OneNote convert documents to OCR, so I decided to test them both using the test clip above.

It took well over an hour for OneNote to convert it to a searchable text, but EverNote has yet to do so even a day later!

Once made searchable, there is a way to create a copy in EverNote, a copy that can then be put in Scrivener, but it’s weird and basically unreadable, showing every word as a separate object.

In OneNote, once the document has gone through the OCR treatment, it’s possible to easily create a searchable text version. (Control-click the document and select “Copy text from picture.”)

This is what I got from my test clip:

Here comes old Woodcock, the Yeoman of Kent, that’s half Farmer and half Gentleman; his horses go to the plow all week, and are put into the coach o’ Sunday.

Tunbridge Walks or the Yeoman of Kent, act I, sc. 1

Not as pretty as ABBYY FineReader, but not at all bad. (I did clean it up a bit.) This text can now be copied and pasted into Scrivener or wherever I want it.

Note: It would have been nice to be able to send this searchable text directly to Scrivener. I passed on this recommendation to OneNote and discovered 1) that their help menu actually helps (EverNote Help is extremely basic), and 2) that they ask how to improve. What a concept! (But do they listen? That remains to be seen.)

A word about Web Clippers

One beautiful thing about EverNote is its Web Clipper. With it, I can send the contents of any webpage to EverNote and, at the same time, indicate which notebook it should be filed in and how it should be tagged.

Evernote Web Clipper 6 For Googles Chrome Browser Launches ...

OneNote’s Web Clipper is not functional on Safari right now due to recent OS changes at Apple. I trust that this will be solved. In any case, it is available on Chrome or FireFox.

It’s a good clipper, but it’s not as useful as EverNote’s. Although you can choose what OneNote notebook to file it in, you can’t specify beyond that with tags, and you can’t file it in more than one place.

OneNote Web Clipper updated with YouTube support, preview ...

Which brings me to Tags

Being able to add tags to a document in EverNote is great. For example, I’d be able to tag an 18th-century French recipe for roasted swans as 18th century, France, food, recipes and swans. This would allow me to narrow a search for a perfect detail regarding a roasted swan snack.

OneNote doesn’t have a tag function, alas—at least not that I can see.

What about cost?

I use EverNote heavily, so I need their Premium plan, which costs $5.83 US a month when paying annually. For that I get 10GB uploads per month, and am able to search PDFs. (For more information about Evernote pricing, click here.)

OneNote is included in an Office 365 subscription package. (Some claim it’s also now available as a free stand-alone, but I’ve not been able to confirm that.) Since I’m already subscribed to the Office 365 world, I can start using OneNote at no additional cost. With OneNote, I get unlimited uploads, so win-win.

Say what? A scanner app?

Scanning pages from books is too slow to be practical. I’m delighted with the Microsoft app Office Lens, which will send a image directly to OneNote. This will save me lots of time.

For example, I took the image below with Office Lens and sent it to OneNote at 10:30 am. In under 30 minutes, it was searchable and even the all-text extract was surprisingly good.

EverNote or OneNote or … ? My conclusion

I do need a database, but given the pros and cons of OneNote and Evernote, where do I stand?

Because of the expense and inconsistent, slow and inadequate OCR function of EverNote, I have decided to migrate my extensive EverNote database to OneNote.

I should mention, as well, that there are indications that EverNote might be heading into hard times, and I don’t want to be left in the lurch.

It’s possible to import EverNote documents into OneNote using their OneNote Importer app, but judging from this note—

The importer software described on this page is still available for you to download and use, but we’re no longer actively developing or supporting this tool.

—that may not always be possible, so migrating now is perhaps wise.

I’ve never been a Microsoft fan—Mac users aren’t their priority—but OneNote for Mac looks worthy, so I’m going to make the move.  I’ve also purchased ABBYY FineReader Pro, and given that I will be unsubscribing from EverNote, I’ll be coming out ahead in more ways than one.  :-)


The links below might be of interest.

Be aware that there are differences between OneNote for Mac and the mothership OneNote for PC users. Also, OneNote for Mac has been recently “updated”—but the changes have caused quite an uproar because it’s no longer possible to arrange tabs along the top, as in this example:

I would love to have such tabs back and I’m hoping the OneNote engineers succumb. Some long-time users are even advocating reverting to the 2016 version and vowing never again to upgrade.

Evernote vs OneNote: The Best Note-taking App in 2019

Top 10 things you didn’t know about OneNote

Using Onenote for your Novel I was excited about trying out this template but it’s for an old version of OneNote, and possibly not applicable to Mac.

Why OneNote is One-Derful for Writers. Inspiring!

On outlining and constructing a timeline/calendar

On outlining and constructing a timeline/calendar

In preparation for NaNoWriMo‘s blast-through-a-first draft-November, I’m following K.M. Weiland’s roadmap on constructing an outline. I’ve outlined my last three novels, but each time it’s like starting from scratch. In any case, I like learning something new.

I began last winter with Weiland’s software, Outlining Your Novel, which is the computer version of her workbook by that name.

I didn’t like it initially, but I dug it out recently, and I’m finding it useful this time around. I’m pairing it with Weiland’s (free) Scrivener template, which in turn is paired with her book:

Children's Publishing Blogs - How To blog posts .

All this to say that reading her book this morning, she mentions using old calendars to establish a timeline for a novel. I’ve been working all month on a detailed timeline on Scrivener, but using printout calendars is an excellent idea. Ms know-it-all Google directed me to this useful site, and I now have color-coded printouts of the years 1549 to 1559 with the significant events highlighted. I especially like that I will be able to know the phases of the moon since this was all-important pre-electricity.


Four other blog posts on outlining that may be of interest:

The advantages of phase or scene outlining

The joy of being at the beginning of writing a novel

How to begin to write a book

A writer’s routine: on resisting an outline

Ta dah! The paperback edition of The Game of Hope

Ta dah! The paperback edition of The Game of Hope

I’m pleased to announce the beautiful Canadian paperback edition of The Game of HopeIt’s fresh and fun to have a new cover. The first person to email me* a selfie holding the book will be sent an autographed hardcover edition.


In other news, I’ve just returned from a three-week trip to the UK, researching the early life of Queen Elizabeth I and the village of Adisham, where I’ve set my other heroine, young Molly the falconer.

Researching falconry at The Hawking Centre was a highlight of my trip.

 

In one month, shortly after Canadian Thanksgiving, Richard and I will be heading south to San Miguel de Allende for the winter. Once settled, I plan to NaNoWriMo-write the rough first draft of Molly & Bess (working title). I’m not yet sure if it’s one novel or two. This will be one way to find out.


*sgulland AT sandragulland DOT com

“Exceedingly pleasant and healthy” — Discovering the village of Adisham

“Exceedingly pleasant and healthy” — Discovering the village of Adisham

One of the most challenging things for me in writing a YA novel based on the scant (and most likely apocryphal) stories “Mary of Canterbury” has been figuring out where to place her. I needed to find an old village in the countryside close to Canterbury and not far from the cliffs of Dover. Proximity to the Pilgrim’s Way of Chaucer fame would be a plus. Also, because of how my story was evolving, I needed proximity to a pond.

I had originally thought that I would “simply” fabricate such a village, but I discovered that that was far from simple—at least for me. It appears that I need a real place to dig into. Ironically, without facts, I am creatively lost.

In researching the turbulent years leading up to the accession of Queen Elizabeth I, I learned of a tiny village not far from Canterbury that was rife with conflict. Like a story-seeking missile, I had found my village.

Adisham (pronounced—I thinkAD SHAM), is an old village not far from Canterbury, not far from Dover, and not far from one of the Canterbury Pilgrims’ paths. And it had had, in former times, a “dangerous pond.” How good was that?

The more I learned about Adisham, the more fascinating it became. A poltergeist in a house near the church? A witch dunked in the pond? A main street called “The Street”?

The biggest bonus was the discovery of John Bland, Protestant rector of the church of Adisham.

A “Canterbury Martyr,” John Bland was one of the first to be burned alive at the stake under the rule of Elisabeth I’s half-sister, “Bloody” Queen Mary. It is also claimed, likely falsely, that he was 103 years old when executed!

I’m about to embark on a research trip to the UK and will be visiting Adisham, talking with people who live there. I’ve already learned that they warn new rectors of what happens to those who run afoul of the churchwarden and the people of the village. :-)


Here are two links on Adisham:

This one shows numerous photos of the church, along with historical details.

Here is a link to a description of the parish, published in 1800, opening with the charming words: “This parish lies exceedingly pleasant and healthy … “

On beginning (again) with Sarah Waters

On beginning (again) with Sarah Waters

In Canada, I have a tall narrow bookcase of books—one of many I have in our house. This one includes poetry, novels I’m either reading or would like to read, and an embarrassing number of books on writing. I am a collector, apparently, a collector of books on writing.

This morning, as I was drinking my delicious mug of decaf, I took three black binders down from the top shelf. I was curious: what were they?

One was a collection of printouts of writing exercises by the New York agent Donald Maass. Another, a thick, heavy binder, was labelled Truby. In it were printouts from master story guru John Truby. I have a lot of Truby—including a series of tapes and his book The Anatomy of Story (which overwhelms me at the first chapter every time I open it). I recalled that at one time Truby offered interactive story analysis on his website; I think it was free, an amazing offering. All the printouts were from his website.

The third binder, labelled Story Tools, was of a middling size. The first page was a list of Sarah Waters’ instructions on how to write a historical novel. Her wise words are no longer online—at least not that I can find—so here it is, my gift to you. (Click here to see the full pdf.)

In the corner I had written: 6 mos min to write 180,000 words.

I wondered when I had written that note. The second page in the binder gave a clue.

Notes before putting away.

I must have written this after I’d been offered a contract to write the Josephine B. Trilogy. Several months before I had finally completed an acceptable draft of The Many Lives & Secret Sorrows of Josephine B., the first novel in the Trilogy. While my agent was looking for a publisher, I had started work on what was to become, nearly two decades later, Mistress of the Sun. On signing a contract for a trilogy, I reluctantly put the project away.

So: all this was Very Long Ago, as I was setting out on this 32-years-and-counting writing adventure.

Sarah Waters’ advice on how to write a historical novel is a treasure. I’ll be returning to it.


The photo at top is of Sarah Waters, 2010, by Sam Jones, as seen in the article in the Guardian on Sarah Waters’ 10 rules for writers (rules which are, of course, spot on).

A Practical Handbook for … writers?

A Practical Handbook for … writers?

One of the books I have in San Miguel is A Practical Handbook for the Actor, by Bruder, Cohn, Olnek and Pollack. It was a useful book to consult when writing about actors in The Shadow Queen, but it’s now and again also mentioned as a useful book for writers. This morning, I scanned through it, before returning it to the shelf.

It’s true that many of the passages are relevant to writing. I especially like this one, for example:

The only talent you need to act is a talent for working—in other words, the ability to apply yourself in learning the skills that make up the craft of acting.

With writing in mind, it becomes: The only talent you need to write is a talent for working—in other words, the ability to apply yourself in learning the skills that make up the craft of writing.

So true!

Here are some others:

You must understand that acting, like carpentry, is a craft with a definite set of skills and tools. By assiduously applying your will to the acquiring of those skills and tools, you will eventually make them habitual.

It’s as if I …

I found Chapter 2, on Analyzing a Scene, particularly useful. As writers, we “experience” the scene we’re creating. These craft tools for actors are useful for writers as well:

  1. What is the character literally doing? (i.e. Tackling a man.)
  2. What is the essential action of what the character is doing in the scene? (The “essential” action is the purpose behind the literal actions, i. e. Physically stopping a horrendous thing from happening.)
  3. What is that action like to me? It’s as if … (For example, “It’s as if I were trying to stop my baby from being killed.” This third one is key to being able to put yourself emotionally into the character.)

The second step of defining the essential action can have a number of possible answers, and each one will require a different response to the third.

For example, it could go like this:

  1. What is the character literally doing? (i.e. Tackling a man.)
  2. What is the essential action of what the character is doing in the scene? (Hoping to get a promotion by proving herself the hero of the hour.)
  3. What is that action like to me? It’s as if … (“It’s as if I would be fired if I didn’t prove I could do a man’s job.)

Just tell the story!

Here’s another sweet spot:

The crucial thing to remember is that the actor is not on-stage to have an experience or to expose himself to the audience, but to help tell a story.

Converting this to writing: The crucial thing to remember is that the writer is not writing in order to have an experience or to expose himself to the readers, but to help tell a story.

Another bit of advice I like is:

Choose something that makes you really want to act the action you have chosen for the scene. … The better your as-if, the stronger your action, and thus the full strength of your humanity will be brought to the work.

The through-line

The through-line is the one action that all the individual actions serve.

You must decide what your ultimate goal is and then construct each individual action to bring you a step closer to achieving that goal.

That through-line is often difficult to discern when it comes to writing a novel. Margaret Atwood once described it as the skewer running through the meat and veggies that make up a shish kebab. That’s a very useful image to keep top of mind.