When I’m researching and I see a bit of text relevant to my WIP (either online or on Kindle), I will clip it: shift-command-4 and “pull” rectangle around the text I want.
It’s so easy to do I often end up with a mess on my computer desktop. Here’s a screen shot to give you an idea how things can look at the end of a day:
My next step is to gather up all the clips and drag them into an empty “Today’s clips” folder. That, at least, looks manageable.
Why it’s important to a researcher for all files to be searchable
Research clips are not of much use to me unless I can search the text. For example, if I was looking for clips about Mother of the Maids I should be able to search that title and all relevant files would be listed.
Clips are a png file, a type of photo, so how is this done?
I’ve messed around with OCR software quite a bit over time. Software such as OneNote claims to convert files automatically, but sometimes so very slowly it’s not practical. Evernote can be touchy and deliver poor results. Also, I need to be able to easily move the OCR’d clip into my writing programmes (Scrivener and AEON Timeline, at this time), which I can’t do — at least not easily — out of Evernote or OneNote.
Why not send a clip directly to Scrivener?
One of the strengths of Scrivener is that I can clip a bit of text from a website directly into my project. However, Scrivener does not offer OCR (at least at this time), so while I will often post a web address URL to Scrivener, I won’t send a clip. I need to be able to search the text of a clip, and without OCR, I can’t do that.
ABBYY FineReader software to the rescue
FineReader is simply an awesome OCR software: fast, powerful and reliable, but at around $150 for Mac, it’s not cheap. (There is a 2-week free trial offered, however.) (Check: is it subscription for Pro?)
I’ll show you what FineReader does with a day’s pile of clips:
I select and drag all of them into the Fine Reader icon in the dock:
It bounces around a bit as it does the work, and then, snap, it’s done.
First I get a “Completed” notice with problem alerts — these I simply ignore and click “Close.”
The next screen shows small images of the clips on the left with enlarged versions on the right. I also ignore all that and click “Export” in the header.
That brings me to this window, which has the PDF option lightly highlighted. I ignore all the options in the middle and click Next in the lower right corner.
The final window gives me the opportunity to name the collection. I type in “Today’s clips” and select where I want them to show up on my computer (desktop, for now).
The really, really, really important thing about this final window is to save each clip to a separate file. (Unless it’s a book file: You don’t want a separate file for each page!)
Click “Export,” and it’s done.
Done? Almost …
I now have 24 searchable clips which I rename and file to put into either AEON Timeline (for events and date-specific clips) or my Scrivener project file. I then trash the original unsearchable clips.
I do a computer search for “Mother of the Maids” and of the over 20 pdf files listed, my 3 newest clips that mention “Mother of the Maids” are listed at the top:
And that’s it, for now. Next up, I hope to write a blog post about Mrs. Stonor (sometimes Stoner), a Mother of the Maids who looked after the young, unmarried Maids of Honour who served most if not all of Henry VIII’s six wives. Four of these maids became the next wife and all four ended up dead.
Now there’s a story. :-)
After not using FineReader for a time, the documents would not come out searchable. I messed with this for hours without luck — until I clicked the “set to default” button in the last frame. :-)
The one problematic part of this process is that the file will come out “untitled,” so I select and copy the title before putting it through the process.
It has been so long since I posted here I couldn’t figure out how to do it. My last post was at the end of October of last year, so close to six months ago. It feels more like a year to me, in part because of our molassas-slow new reality.
I’m having what I consider a non-productive day: I’ve not written or edited a word. Resistance rules!
I’m fond of the French word flâneur, meaning an idle person who strolls about without object, who putters around, in other words.
Paul Gavarni, Le Flâneur, 1842
I’m especially fond of the verb form flâner, which means to stroll. Flânerie is the act of strolling, and that’s the kind of day I’m having. Distracted, easily side-tracked by shiny objects, unofficially off-track.
Or is it?
One of the things I came upon this morning were the animated portraits I made some time ago through My Heritage. It’s meant to allow people to animate photos of family members (which I found just a little creepy), but enchants me when used to animate historical portraits.
For example, based on a close-up clip from this portrait of Princess Elizabeth …
ATTRIBUTED TO WILLIAM SCROTS, Elizabeth I when a Princess c.1546
… I made this charming animation:
Needless to say, I then had to make animations of most of my cast of characters.
A word of caution however
“Flaneuring” (our household verb) can also lead to overwhelm. In rediscovering these animations, I found I had computer folders of desktop contents five layers deep. It’s like an archeological dig.
I’ve been stuck for nearly a week over a chapter in the WIP. (The whip, I think ruefully, as I type those letters.) The problem has many causes. One is that I have a stubborn need to know where-the-heck my heroine (Elizabeth Tudor, in this instance) is, in fact. It’s a period of only two days, and historians don’t provide the details—which should lead me to suspect that the information simply isn’t available.
Edward VI with flowers by William Scrots, circa 1550
It’s an important moment, so I’m surprised not more is known. Fifteen-year-old King Eddie VI has died, and (after something of a bloodless battle) his half-sister Mary has been proclaimed queen.
Portrait of Queen Mary I of England by Antonis Mor, 1554.
Mary’s much younger half-sister Elizabeth (not yet twenty), is now the heir to the throne. She is riding out to meet Mary—to bow before her sister queen.
This website account is fantastic, but this one date is unlikely, in my view, because Mary was proclaimed queen in London on July 19.
According to historian Tracy Borman in Elizabeth’s Women (page 136), Elizabeth wrote Mary on that day to congratulate her, but also …
Showing all due deference, she also humbly craved Mary’s advice as to whether she ought to appear in mourning clothes out of respect for their brother, Edward, or something more festive.
(This is the type of detail I relish.)
It would have taken time for Elizabeth’s missive to reach Mary, for she was in the northeast, at her Framlingham castle, already attending to matters of state business and debating whether or not to go to London. Some advised her that it would be wise to return soon while the public was so enthusiastic about her. On the negative side, it was stinking hot in London and there were rumours of plague.
Mary was apparently prepared to be magnanimous in her triumph. She therefore invited Elizabeth to accompany her to London. (Borman’s Elizabeth’s Women)
Mary set out for London on Monday, July 24. It was a long journey: Ipswich (two nights), Colchester (one night), Newhall (three nights), Ingatestone (two nights), Havering (one night), finally arriving at Wanstead House on Tuesday, August 1, where she welcomed Elisabeth the next day, on August 2. (This overnight stay is rarely mentioned in biographies.) Together they set out for London on Thursday, August 3—with a combined entourage of over twenty thousand—arriving late that afternoon in London.
Elizabeth set out to meet Mary on Saturday, July 29, and by most accounts, she stayed for only one night in London before heading out the next day, Sunday, July 30, to meet Mary.
The journey from the London gate to Wanstead takes but a few hours on foot. (See my note below.) If Elizabeth was in London on July 29 for only one night, and met Mary on August 2 near Wanstead, where was she on July 31 and August 1? She was travelling with an entourage of over a thousand, so it was not as if she could drop in just anywhere.
This question foolishly cost me several days of work. I finally found support for the likelihood that Elizabeth had simply stayed at Somerset House, her new (to her) manor in London, for the full three nights. (Elizabeth I; The Word of a Prince, by Maria Perry, page 83.)
Second, the frazzled meet-up
A few historians state that Elizabeth stayed with Mary for one night at Wanstead House, and the consensus seems to be that they met on the road, and that Elizabeth dismounted and knelt in the dirt before Mary.
The problem with writing fact-based fiction — at least for me — is that things have to make sense. So now my question was: If Mary was expecting Elizabeth at Wanstead House, why did she meet her on the road? I didn’t want to spend another week on this, so I decided to sketch out a draft where they meet on the road, Elizabeth kneels, and they move on to Wanstead House from there.
But then what?
However Elizabeth and Mary meet, the sheer size of their entourages boggled my mind. Elizabeth had an entourage of over a thousand, but it was nothing to compare with her sister’s following. Imagine:
In the late afternoon of 3 August, Mary Tudor set out in procession from Wanstead to take possession of her kingdom. Those who stood along the processional route to London were astounded by the great number in her party. Mary had an escort of some ten thousand people with her – ‘gentlemen, squires, knights and lords’, and not to mention, the various peeresses, clergymen, judges, heralds, and foreign dignitaries come to pay her tribute.
—The Turbulent Crown: The Story of the Tudor Queens by Roland Hui (p. 322).
This is an image of the coronation procession of Elizabeth’s brother, King Edward VI. Imagine the traffic congestion!
This was at a hot time of year: the dust clouds of Mary’s procession coming through rural Essex must have been horrendous.
It did not take long to run into another time-consuming research question: Once they reach London, followed by well over ten thousand, what route do they take to the Tower of London? I decided that one hint might be to find out what the traditional route for a regal procession to or from the Tower of London might have been. This (eventually) led me to an amazing book (in four volumes): The Progresses and Public Processions of Queen Elizabeth by John Nichols on https://books.google.ca. On page 115 of Volume 1, there is this map:
Knowing this, I was able to chart the route on the fabulous interactive Agas Map of Early Modern London: Entering at Aldgate, they would wend their way down Aldgate Street, take the left fork onto Fenchurch Street, left onto Mark Lane, right on Tower Street and right again on Petty Wales to the entry into the Tower.
To the Tower!
My second puzzle, also solved today, was to determine where were the queen’s apartments in the Tower of London. Several maps later, including the one below, I discovered that the queen’s apartments were in the lower right-hand corner of the Tower premises, fairly cut off from any unpleasantness.
In addition to great details, the site provides this image of what the Great Hall at the Tower likely looked like:
This helps give me a feeling for what the rooms beyond might have been like.
How much of this is likely to end up mentioned in the novel? Likely very little, but knowing what’s what helps me imagine the scenes.
Confession: two research tricks
To estimate approximate walking distances, I find it useful to use maps.google.com.
(Too bad maps.google doesn’t have an “on horseback” option. For this, I suspect that somewhere between “walking” and “by bike” might be an approximation, given all the stops horseback travel requires to give the horses rest, food, and water, or possible exchanges.)
Another part of knowing what’s what is determining when the sun rises and sets, and (particularly in this time) when the moon is full. I can’t track that for 16th century England, so instead, to at least keep the sun and moon on realistic trajectories I’m using the current calendar for the UK using this fantastic site: timeanddate.com.
And so? So now “all” I have to do is write the #%&@ scenes.
The novel I’m writing now is set in mid-16th century England. During this time period episodes of black plague and the quickly lethal “sweating sickness” came and went. With each epidemic, enormous numbers of people died.
Long ago, when I started to research, these events were simply blips on a timeline. With the advent of our Covid-19 world, such facts became far more vivid to me. I hadn’t understood the fear and heightened state of caution epidemics caused.
A 16th-century story to set the stage: a man and woman in a village in England lost children to the plague. Another child was born, and when plague returned to their town, they sealed shut the windows and doors of their home. Thanks to their precautions, their child survived: his name was William Shakespeare.
Shakespeare wrote “King Lear” (and “Macbeth,” and “Antony and Cleopatra”) during plague years when the London theatres closed down. (The rule was that once the death toll went over 30, playhouses had to close.) In short, he was out of work and had time on his hands.
“King Lear” is one of his bleakest plays, written while living in a bleak time:
The mood in the city must have been ghastly – deserted streets and closed shops, dogs running free, carers carrying three-foot staffs painted red so everyone else kept their distance, church bells tolling endlessly for funerals … (The Guardian, March 22, 2020)
Plague also changed the nature of the plays he wrote. Plague killed off men in their 30s, so the demographic of both his actors and audience changed.
Although A Journal of the Plague Year by Daniel Defoe is not, in fact, a contemporary account—Defoe was a master of what I would call fact-based fiction—it is thought to have been well-researched. I was struck, reading it, how well-organized England was in combating epidemics. For example, if infected, people were prevented from leaving their homes. One needed a certificate of health in order to travel. Interesting!
Certainly, it is reminiscent of what we are going though today:
City authorities are sane and composed concerning the spreading plague, and distribute the Orders of the Lord Mayor and Aldermen of the City of London. These set up rules and guidelines for the arrangement of searchers and inspectors and guardians to monitor the houses, for the quieting down of contaminated houses, and for the closing down of occasions in which enormous gatherings of individuals would assemble.
Here’s a truly contemporary word of caution from 1665:
This poem by U.S. poet Daniel Halpern was published—astonishingly—seven years ago in Poetry Magazine. (Likewise astonishingly, he doesn’t remember writing it.)
There are fewer introductions
In plague years,
Hands held back, jocularity
No longer bellicose,
Even among men.
Breathing’s generally wary,
Labored, as they say, when
The end is at hand.
But this is the everyday intake
Of the imperceptible life force,
Willed now, slow —
Well, just cautious
In inhabited air.
As for ongoing dialogue,
No longer an exuberant plosive
To make a point,
But a new squirrelling of air space,
A new sense of boundary.
Genghis Khan said the hand
Is the first thing one man gives
To another. Not in this war.
A gesture of limited distance
Now suffices, a nod,
A minor smile or a hand
Not in search of its counterpart,
Just a warning within
The acknowledgement to stand back.
Each beautiful stranger a barbarian
Breathing on the other side of the gate.
I’ve been putting research documents into Scrivener, assuming that they were searchable. After all, one oft-stated advantage of using Scrivener is that you have all your documents in one place.
It’s true that I can put everything and anything into Scrivener, but I also need to be able to search within those documents. I mistakenly assumed that one of Scrivener’s many superpowers was the ability to make all documents searchable. In other words, I assumed that Scrivener utilized OCR (Optical Character Recognition). Not so. :-(
Having searchable documents is important for my current WIP because it’s set in the 16th century, and a number of the resources are rare and/or ancient and only available on BooksGoogle or InternetArchive. I’ve taken to clipping relevant parts of such documents (shift-control-4 on a Mac) or exporting them whole as PDFs before sending them to Scrivener. The clips are a type of image, so they need OCR to be searched, and most PDFs are not searchable as well.
And so I began to look at ways to make documents searchable before putting them in Scrivener. In the process, I discovered that anything to do with OCR opened a bottomless pit. I will try to keep this simple.
Dedicated OCR software
One possibility would be to invest in a software programme dedicated to making documents OCR searchable. The highest-rated programme for Mac is ABBYY FineReader Pro, available on trial for 30 days. I tested it out on a clip (below), and in seconds had a searchable Word document that beautifully preserved the formatting of the original.
This is the original clip:
And this is the searchable Word document:
Databases that make documents searchable
The other possibility would be to use a database that automatically makes documents OCR searchable. The advantage of using such a database is that it is—duh—a database, a logical place to store research documents. … which brings me to OneNote and EverNote.
Both EverNote and OneNote convert documents to OCR, so I decided to test them both using the test clip above.
It took well over an hour for OneNote to convert it to a searchable text, but EverNote has yet to do so even a day later!
Once made searchable, there is a way to create a copy in EverNote, a copy that can then be put in Scrivener, but it’s weird and basically unreadable, showing every word as a separate object.
In OneNote, once the document has gone through the OCR treatment, it’s possible to easily create a searchable text version. (Control-click the document and select “Copy text from picture.”)
This is what I got from my test clip:
Here comes old Woodcock, the Yeoman of Kent, that’s half Farmer and half Gentleman; his horses go to the plow all week, and are put into the coach o’ Sunday.
Tunbridge Walks or the Yeoman of Kent, act I, sc. 1
Not as pretty as ABBYY FineReader, but not at all bad. (I did clean it up a bit.) This text can now be copied and pasted into Scrivener or wherever I want it.
Note: It would have been nice to be able to send this searchable text directly to Scrivener. I passed on this recommendation to OneNote and discovered 1) that their help menu actually helps (EverNote Help is extremely basic), and 2) that they ask how to improve. What a concept! (But do they listen? That remains to be seen.)
A word about Web Clippers
One beautiful thing about EverNote is its Web Clipper. With it, I can send the contents of any webpage to EverNote and, at the same time, indicate which notebook it should be filed in and how it should be tagged.
OneNote’s Web Clipper is not functional on Safari right now due to recent OS changes at Apple. I trust that this will be solved. In any case, it is available on Chrome or FireFox.
It’s a good clipper, but it’s not as useful as EverNote’s. Although you can choose what OneNote notebook to file it in, you can’t specify beyond that with tags, and you can’t file it in more than one place.
Which brings me to Tags
Being able to add tags to a document in EverNote is great. For example, I’d be able to tag an 18th-century French recipe for roasted swans as 18th century, France, food, recipes and swans. This would allow me to narrow a search for a perfect detail regarding a roasted swan snack.
OneNote doesn’t have a tag function, alas—at least not that I can see.
What about cost?
I use EverNote heavily, so I need their Premium plan, which costs $5.83 US a month when paying annually. For that I get 10GB uploads per month, and am able to search PDFs. (For more information about Evernote pricing, click here.)
OneNote is included in an Office 365 subscription package. (Some claim it’s also now available as a free stand-alone, but I’ve not been able to confirm that.) Since I’m already subscribed to the Office 365 world, I can start using OneNote at no additional cost. With OneNote, I get unlimited uploads, so win-win.
Say what? A scanner app?
Scanning pages from books is too slow to be practical. I’m delighted with the Microsoft app Office Lens, which will send a image directly to OneNote. This will save me lots of time.
For example, I took the image below with Office Lens and sent it to OneNote at 10:30 am. In under 30 minutes, it was searchable and even the all-text extract was surprisingly good.
EverNote or OneNote or … ? My conclusion
I do need a database, but given the pros and cons of OneNote and Evernote, where do I stand?
Because of the expense and inconsistent, slow and inadequate OCR function of EverNote, I have decided to migrate my extensive EverNote database to OneNote.
I should mention, as well, that there are indications that EverNote might be heading into hard times, and I don’t want to be left in the lurch.
It’s possible to import EverNote documents into OneNote using their OneNote Importer app, but judging from this note—
The importer software described on this page is still available for you to download and use, but we’re no longer actively developing or supporting this tool.
—that may not always be possible, so migrating now is perhaps wise.
I’ve never been a Microsoft fan—Mac users aren’t their priority—but OneNote for Mac looks worthy, so I’m going to make the move. I’ve also purchased ABBYY FineReader Pro, and given that I will be unsubscribing from EverNote, I’ll be coming out ahead in more ways than one. :-)
The links below might be of interest.
Be aware that there are differences between OneNote for Mac and the mothership OneNote for PC users. Also, OneNote for Mac has been recently “updated”—but the changes have caused quite an uproar because it’s no longer possible to arrange tabs along the top, as in this example:
I would love to have such tabs back and I’m hoping the OneNote engineers succumb. Some long-time users are even advocating reverting to the 2016 version and vowing never again to upgrade.
In preparation for NaNoWriMo‘s blast-through-a-first draft-November, I’m following K.M. Weiland’s roadmap on constructing an outline. I’ve outlined my last three novels, but each time it’s like starting from scratch. In any case, I like learning something new.
I began last winter with Weiland’s software, Outlining Your Novel, which is the computer version of her workbook by that name.
All this to say that reading her book this morning, she mentions using old calendars to establish a timeline for a novel. I’ve been working all month on a detailed timeline on Scrivener, but using printout calendars is an excellent idea. Ms know-it-all Google directed me to this useful site, and I now have color-coded printouts of the years 1549 to 1559 with the significant events highlighted. I especially like that I will be able to know the phases of the moon since this was all-important pre-electricity.
Four other blog posts on outlining that may be of interest: