Rises In The West

Have you ever had the experience where you see a word so many times in a day that it no longer seems to be a word? The correct spelling doesn’t seem correct. I feel this on occasion and it’s always strange.

North and south are easy to picture on my mental map of the world. North is up, south is down. When it comes to east and west, however, I use a mnemonic to remember which is which. We. West is on the left, and East is on the right. It feels silly to admit this, but it helps me remember.

There’s also a saying to remember in which direction the sun rises and sets. I’m sure many know it. Can you recall what it is? This phrase came to my mind when I wanted to mention the rising sun in a story.

I pictured a flat map of Earth with the cardinal directions printed beyond the edges of page. N E S W. And I said to myself, “Rises in the west and sets in the east.” This fits nicely into my mental model of reading from left to right and thinking of directions from left to right, as I mentioned for We.

Except I was wrong. The sun actually rises in the east and sets in the west. A day or two ago, as I was doing some unrelated editing of the story, I had a sneaking suspicion I’d mixed something up. Had I gotten the direction of the sunrise backward? I sure had. Even after searching to verify which phrase was right, it didn’t really click. It feels like a word that doesn’t look spelled correctly. Okay, I can get it if I really think about it, but it doesn’t feel right. The wrong phrase rolled off the tongue so easily — “Rises in the west, sets in the east” — even though it was backward. That’s a bit worrisome. How many times does my mind play tricks on me and I don’t realize it? I’m glad I caught this instance before someone else did.

This is a reminder to double check my work on what seem to be trivial problems. Otherwise, 1 + 1 might end up equaling 3 because it just seems right in my weird, little mind.

Method Delegation In Ruby

When creating a new Ruby class, it can be helpful to use composition over inheritance. Let’s look at an example of an Image. Our Image has a File, but we don’t want the Image to extend File. Why not? We don’t want Image to have the full interface of File; just a small subset of it. This is a perfect use case for composition and method delegation.

Here’s what our Image class looks like right now. It contains the File object we’re interested in.

class Image def initialize(image_path) @file = File.open(image_path) end end

Next, we would like to expose a few of the File methods on Image: #path, #read, and #close. One way to do that is by writing our own methods that pass the message along.

“` require ‘forwardable’

class Image def initialize(image_path) @file = File.open(image_path) end

def path @file.path end

def read @file.read end

def close @file.close end end “`

This quickly gets repetitive and verbose.

Fortunately, Ruby provides an easy means of delegating methods to other objects. We’ll extend the Forwardable module, and then use def_delegator to hook up the delegation.

“` class Image extend Forwardable

def initialize(image_path) @file = File.open(image_path) end

def_delegators :@file, :path, :read, :close end “`

We’ve accomplished the same functionality in fewer lines, and made it more readable.

Unfortunately, I find Ruby’s core method delegation unintuitively named and implemented. It’s confusing to remember the Forwardable module gives us method delegation, and that the method is named def_delegators. I wrote this small post so I can refer to it later, when I need a refresher on how to do method delegation in Ruby.

The Novel and The Myth

In 2015, I had my most productive year of writing. I wrote with some frequency on a variety of topics. Overall, I published 64 posts to this blog. It felt fantastic to accomplish that! However, I haven’t been as active on my blog in the last few months. What happened?

The truth is, I have been working on my writing even more regularly – most days. The difference is that my focus is different and the format is not blog posts. The result being that publicly visible portions don’t yet exist. It feels a little strange to go from sharing writings regularly to not doing so for months. I wanted to share a little about what I’m working on.

At the end of November, I began writing a novel. I’m calling it “The Long Signal”, and the genre is science fiction. I so far have six chapters and over 17,000 words in the first rough draft. I’m excited about this project and look forward to continuing it! I reached the end of the 6th chapter and decided to work for a while on a tangential project.

This blog has been titled “Thoughts of an Eaten Sun” since I began it in 2009. For a long time, I’ve had an idea of the story behind that phrase. It’s a myth and more in the fantasy genre. I have some brainstormings that are three and a half years old now, but I’d never written anything substantial for it. Now seems like the perfect time to bring it into existence!

My plan is to tie the novel and the myth together. To do that, I wanted to flesh out the myth of “Thoughts of an Eaten Sun”. At the end of December, I began writing it. I recently finished its first draft, and it stands at about 21,000 words. The next step will be to do some revision, editing, and additional writing. I plan to self-publish the myth, which will be a fun and interesting project of its own. And once it’s completed, I’ll move focus back to “The Long Signal”.

“Thoughts of an Eaten Sun” will share some similarities to “The Mechanism Collection” in that they are both mythologies in the same universe. “The Long Signal” is another story in that universe and is set in a more modern time.

My near-daily act of writing over the last couple months has helped spawn even more ideas. I’ve taken many notes and am excited to explore them in the future. Both Zach and Karla have been fantastic with their feedback, so I wanted to be sure to thank both of them.

I’m plenty surprised that I’ve written 38,000 words for these two projects since the end of November. But the encouragement and support of those close to me, along with a regular habit, have helped make writing something I really look forward to.

I’ll lose some of the words-per-day traceability when I go to edit “Thoughts of an Eaten Sun”, but I hope to keep to my goal of at least 300 words or 30 minutes of work on writings most days. Additionally, I have taken days off from writing here and there, but I don’t beat myself up for it. Balance is important.

I look forward to sharing these works with others as I make more progress on them.

Make All Gems Pristine After Upgrading Ruby

I upgraded to Ruby 2.3.0 today after using 2.2.3 for some time. Next, I ran a Ruby script I’d used many times before only to get a terminal full of output like this:

Ignoring iconv-1.0.4 because its extensions are not built.  Try: gem pristine iconv --version 1.0.4
Ignoring json-1.8.2 because its extensions are not built.  Try: gem pristine json --version 1.8.2
Ignoring nokogiri-1.6.6.2 because its extensions are not built.  Try: gem pristine nokogiri --version 1.6.6.2
Ignoring rdiscount-2.1.7.1 because its extensions are not built.  Try: gem pristine rdiscount --version 2.1.7.1
Ignoring redcarpet-3.2.2 because its extensions are not built.  Try: gem pristine redcarpet --version 3.2.2

Looks like gems with native extensions didn’t work after upgrading Ruby. Makes sense, but how could I reinstall all these gems without having to manually run all of those pristine commands? Fortunately, there’s a way to make all your gems pristine.

gem pristine --all

Beware that, if you have made changes to any gems locally, that this will clobber those changes. Thanks, StackExchange, for leading me to the answer.

I was happy to see the solution was easy, even if it did take a few minutes to process all the gems. I’m sure I’ll need to refer to this post again when I upgrade Ruby in the future.

Introducing everything-core Gem

I’ve been using my everything concept for a while now. I enjoy the structure it gives me for my writing collection. To help me do things with the writings, I’ve written various pieces of software. Quickly enough, I had similar code duplicated across multiple projects, with little differences here and there.

Rewriting that code every time or even copying and pasting it into each new project isn’t an extensible way to build a suite of tools. Instead, I want a single piece of code I can include in multiple projects and just use. I also wanted to explore creating gems for Ruby. The goal is to keep the core software pretty small, and then extend it using other, small gems.

To that end, I’ve been working on everything-core. It provides the basic functionality needed to work with a piece in everything.

I haven’t used it much on my own yet, but that’s because only tonight did I finally get enough functionality in there to make that feasible. Now, you’re able to get a piece’s markdown content as well as its metadata. I’ve also tested the code in the gem, to make things a little more robust. I’d appreciate if you report any issues you run across.

I’m writing this post, in part, to check whether I can rework my WordPress publishing script to use the gem. If this post makes it live at all, it will be proof of success!

I’m happy to share a small milestone in my project. I look forward to expanding on this core going forward.

Directions in the Mountains

I just finished reading my friend Natasha’s post Sometimes, The Journey Is Hard. Her experience with bad routing directions in the mountains reminded me of my own.

Fool Me Once

Several years ago, I was trying to get to the campground at Grays and Torreys. I’d been there a time or two before, but it had been a while. I did what I normally do for directions, and found the area on Google Maps. Then we set out, with that as our destination. The area of the highway exit seemed about right. The dirt road we drove up wasn’t familiar, but I chalked that up to bad memory.

As we went along, the road condition deteriorated. I recall the road up Grays and Torreys being a bit rough in areas, but this had large gouges across the way. It quickly became a 4×4 road, which I navigated in my 2WD sedan. I had to attack the road’s chasms at sharp angles and low speeds to avoid bottoming out or becoming stuck. It can’t last forever, I thought, and we’ll soon be at the campground.

Except we ran out of road. The dirt road literally ended at a closed, steel gate. No signage; nothing. I then realized I’d taken the wrong road completely. There weren’t any turnoffs I could have missed. We were simply on the wrong road. And now it was dark. We’d lost the last of the sunlight on our way up.

After turning the vehicle around and navigating those same damn ravines and gulches in my poor, poor car, we made our way back to I-70 – and cell service. My car handled the road fantastically, all things considered, even though it did take a small beating.

With a data signal, I was able to check out Google Maps, along with directions on the 14ers site to see what went wrong. Google had taken us to a road on the opposite side of the mountains from where we wanted to be. We’d gone too far west, and ended up in Nowhere.

This floored me. The campground for Grays and Torreys might not have been huge, but it was at least a thing. Whereas the road Google Maps had just taken us up was a dead-end with nothing along the way. How could Google get it that wrong? What about all the other thousands of hikers a year who hike Grays and Torreys? How do they get there? Maybe they followed other directions, or knew better than to trust Google blindly. Worse yet, how many other hikers per year are also lead astray by bad directions?

We had to backtrack to get to the eastern side of these peaks. Finally, we found the correct exit and dirt road, and made our way up to the trailhead, to where we would camp. It was many hours later than expected, in the pitch dark, but we did arrive.

This experience taught me to double-check the route and destination Google will suggest when heading into the mountains. It doesn’t seem particularly knowledgeable of hiking trail heads, even for the most popular of destinations.

Fool Me Twice

Earlier this year, Karla and I did some hiking in the mountains south of Idaho Springs. After the hike, we wanted to get into town and have some drinks and pizza at Beau Jo’s. I put the instructions into Google Maps and it calculated the fastest route to the restaurant.

We headed down a paved road, turned onto a well-maintained dirt road, and followed that for a while. A few minutes later, I was turning the car around because I’d taken a fork other than the one Google instructed. The road I kept to was in much better condition that the one the directions used, but it did say to take it. I didn’t have a signal to get an alternative route, so I pressed on, following Google’s lead. I’m sure you know where this story is headed.

The condition of the road wasn’t great when we started, but it got worse the further down we went. I should have taken this as an indication to turn around immediately, but I thought Google knew better. Because you know how smart their technologies are in other parts of your life, so why would this one be any different?

At a certain point, we reached a CLOSED sign, indicating the way wasn’t maintained in winter. That gave me pause, certainly. Except, it had narrowed so that turning around was impossible. There was no option but to keep going down.

Along the way, little streams flowed over the road here and there. Other parts had enormous rocks to dodge, or bang and scrape your car over. Sometimes, the road tilted us quite a few degrees on our side. A couple areas were so narrow that branches touched the car on both sides. Portions of the path had grass growing where the tires rode. That’s how long it had been since someone else traveled through there.

The entire time, I was waiting for us to reach a spot impassible, and then not be able to turn around. Eventually, I realized that even if we could have turned the car around, there’s no way I would make it up the terrain. Gravity was at least working with us in this direction.

It was freaky to realize this road was a giant mistake. No one would find us if we got stuck. We’d have to hike out however far to find someone. I have AAA, but would they even help if my car was stuck in a place like that?

I had worried about meeting some 4×4 vehicle coming up the road, and being unable to let them pass. But at least if we passed a vehicle going up, we’d know the road led somewhere. As it stood, we passed no one, and had no idea how long the road was.

Further down, we passed a creepy, abandoned trailer which had furniture torn up and scattered around. In the nearby area, there was a lot of scattered property. It looked like a scene from a horror movie.

Shortly after the cabin of despair, we reached a spot where a couple of guys and kids were walking alongside the road. I rolled down the window and said hello. They were exploring some of the junk that lay scattered in the woods.

I asked how they’d gotten up here, and how much further the road went. Turns out they were also in a Honda Accord and tried to come up the road. They’d given up and turned the car back. One of them said, “The worst is yet to come.” My first thought was, “Oh. Shit.”

Luckily, that little part of the road was nothing compared to what we’d been down. The guy had no idea how much worse the terrain was above. I told him there’s no way he could make it up, so he was smart not to press on.

From here, it was smooth sailing into Idaho Springs. Karla and I had made it through the unsettling uncertainty of how long the road was, and if it would just end.

Only after we’d made it out of there did I realize something else. The road we were originally on was Little Bear Creek Road. The road Google took us down was called Old Little Bear Creek Road. The “Old” in the name should have been a sign. It’s named that for a reason.

Funnily, the Little Bear Creek Road that Google didn’t take us down would have been loads faster, without the threat of being stranded. Google certainly didn’t account for the state of the Old road, or the snail pace required as we negotiated it.

In this case, Google had the right endpoint, and the routing was nearly there. But it reminded me to be more cognizant of what roads the directions take. If, in the mountains, one option is a wide, flat, clear, maintained dirt road, and the other is a narrow, grassy, steep, rocky road, use some common sense, Kyle! You’re better off turning around well before the CLOSED sign, instead of blindly following the output of an algorithm.

Better Not Be a Third Time

Fortunately we made it out in both of the above occasions. My car’s been dinged up before, and I don’t care about keeping it pristine and shiny. It certainly wasn’t the end of the world. Even got some good stories out of it.

Google Maps works really well in cities, but it definitely breaks down in the mountains. I hope to keep these trips in mind going forward so I don’t repeat them. I don’t want a worse experience where the only reason it happened is that I didn’t question the directions as I went along.

Start The Writing

I had finished reading Neuromancer while lying on my futon, and then I stayed there a while. I thought over that book, along with some others I’d recently finished that were written by Arthur C. Clarke. An idea for a science fiction story sprouted in that moment. As my mind raced along, I got up, went into my den, and wrote down my brainstormings. The idea surprised me in its intensity and completeness. This, I thought, is something I can turn into a book. There were still lots of details to flesh out, but it had substance.

Over the next few days, I talked to Zach about some of the technical details to make sure they seemed feasible. I better understood the sequence of events after telling him about it. He gave me the fantastic advice to create something like a storyboard. I’m still working on it, but it will be nice to have a high-level summary of the story as I work on it.

I also talked to Karla, and she helped me realize how I had not yet envisioned individual characters. I had vague notions of people involved like “scientist”, but nothing more detailed. We talked about archetypes and personalities that could help form the basis of actual personalities. Thanks to that brainstorming, I have an idea of who the first character will be.

Next comes the part of figuring out how to make progress toward that leviathan goal of having a completed book. It will take dedication, persistence, regularity, and hard work. I want to hold myself accountable to some standard of effort, but what metric should I use to gauge it?

My initial thought is something like 3 pages per day. But pages is an ambiguous term. The size of the paper and print, margins, and layout of the text can vary the number of pages. Additionally, when I write on the computer, there isn’t a real concept of pages. It’s just a flow of words in a text file.

So what about 300 words per day? I’m not exactly sure how many words that would look like when written out. But it feels like a goal that’s reasonable and doable. One that I could meet regularly. 300 words a day for a year would be nearly 110,000 words. That’s certainly a novel’s worth. This goal of words-per-day does not, however, take into account things beyond the act of writing. Editing blog posts, for example, takes quite a bit of time. I can only hazard a guess as to how much time would be required for something the size of a novel.

Karla suggested something like 30 minutes per day. Or an hour per day, three times per week. This seems like a good benchmark, because this will be a long process. I can imagine there being days where I make decent progress in less than an hour, but that’s foolish optimism to assume it’d happen regularly.

I will have to write the words, and I will have to put in the time. So there’s the possibility of combining the two. 300 words or 30 minutes per day, whichever I reach first.

In reality, these will be guidelines. It’s not a science, and I have never done this before to know better. My hope is that these achievable minimums will encourage me to work on it beyond the 300 words or the 30 minutes each day. Momentum building over time.

The purpose is to have measurable ways to ensure I keep taking steps toward my goal. Perhaps as I get further into it, I’ll know what makes more sense, and what works for me. I can imagine the most difficult part of setting these daily goals to be holding myself accountable.

In the past, I tried writing a book called The Emperor’s War, but got stuck in a loop of editing. I had something like six chapters, and spent a lot of time revising them again and again. Trying to make them perfect. Forward progress was minimal. To keep from repeating myself, I want to set another guideline: keep writing forward. I’ll make good progress if I don’t allow editing until I’m done with the first draft. I can find out what the story is before “refining” it.

I now have a general outline of the story, and a glimpse of how to begin it. But there’s an uncertainty of how to actually start the writing. How do I begin it in a way that makes sense, introduces the world and characters, and doesn’t feel too abrupt? I have a scene in mind, but it needs some setup to not feel jarring. I’ve pondered these ideas lately, but the words are still elusive.

I spent most of yesterday mulling it over, and subsequently putting it off. Around 1am, I found the motivation and then hand-wrote two and a half pages. It was nothing like I expected, and I couldn’t have predicted what it would look like, but it was something. I’m okay with how it turned out.

I aim to write more today, but the idea of starting again is daunting. I don’t know what I’m doing. I’m also afraid it’ll turn out like shit. So I put it off some more. This post is a by-product of that procrastination. Nothing like a goal to help motivate you to accomplish everything else on your TODO list.

Anyway, I need to get to the writing and stop putting it off. It won’t be what I imagined, but it’ll be better than nothing. Forward momentum to get a first draft, and then will come the editing to clean it up.

The Downfall Of Formats

During 2015, I wanted to spend more time at home, and cultivate creativity there. Because I’ve enjoyed writing since I was young, I aimed to write more consistently. The frequency with which I’ve published blog posts this year is a signal of my progress. I’m proud of this, and hope to continue this trend next year as well.

My second goal has been to preserve my older writings. What does that even mean? For physical writings that exist on paper, it means digitizing them into a format that has the potential to survive technological upheavals. For writings which are already digital, it means rescuing them from the graveyards in which they’re currently trapped.

On this front, I’ve started preserving my early poetry, and typing up hand-written pieces. Going through this process has brought to mind why this is important to me. Bear with me; this is a weaving exploration of these goals and other topics. I won’t apologize, however. Life is not straightforward, so neither can all stories be.

The Early Software

I first wrote on computers with Microsoft’s operating system, software, and file formats. The first piece I remember creating for school was an article on space. I wrote it in Microsoft Works, and the format is .wps. I spent most of my time, however, using Word – particularly for my early poetry – and that format is .doc.

Word worked really well. I was able to focus on the content and even style the documents easily. It had a lot more features than Works, which was another draw. My school’s computers had Word, which meant I could use a single program in multiple places. That was convenient.

I played around with Corel WordPerfect, but it didn’t catch on. Using another text editor like Notepad never occurred to me. Notepad had zero features compared to Word. Even line wrapping was a pain in the ass.

At home, we had a single computer in the house. The entire family would get time on it, although I’m sure I took up the lion’s share. After PDAs came out, I wanted one badly. Having one with an external keyboard would allow me to write more often and easily. I wouldn’t have to share time on it, nor would I have to type up things originally written by hand. I pictured myself using it all the time. Ahh, to dream. I never did get a PDA, and, as you might have guessed, my life has been a complete train wreck ever since.

Occasionally, I backed my documents up on floppy disks. To back up a piece I wrote about the Titanic, I had to split it onto multiple diskettes because it was too large to fit on a single one. But don’t let me fool you, the images I added to the file were the main culprit. It’d take a large amount of text to fill a floppy disk, and I’m not prolific enough for that.

Enter the Web

I recently posted the story of how I became a web developer, which recounts how my love for writing lead me to my profession. Along with creating my own websites, I looked for other sites I could also use for writing.

Writing.com was a big one. It was a great place to find new things to read, and receive feedback from others. Eventually, I got banned for trying to exploit a loophole on the site. That hurt. A lot. It wiped out all the reputation I’d built up on the site, and I couldn’t connect with people I’d met there. That site later went to crap. I’m not saying that happened because I got banned, but that’s kind of what I’m saying.

I learned of Xanga and used that a decent amount. Other friends like Zach and Fergy had accounts too, and we’d post there with all our latest happenings. I also played around with phpBB as an online bulletin board for chatting with friends. From college onward, I have used other programs to compose and store my writings like OneNote, WordPress, Google Wave, and, most recently, Evernote.

Left Behind

All in all, I’ve used a decent number of applications. My experiences with them over the years have been enlightening. I’ve experimented with different software and workflows, which has helped me find what works for me. It’s also brought to light some problems.

Remember the space article I wrote in a .wps file? Sadly, my latest copy of Word can’t even open it. Microsoft has already abandoned one of its formats. An open source alternative called LibreOffice can open the file, so I’m still able to access it. For now.

The majority of my early writings were .doc files, and they’re still accessible today. Oddly, some of the files don’t render properly in newer versions of Word, LibreOffice, or Google Docs. This makes it a bit challenging to read through them today.

I hated Apple even more than I hated Microsoft, so I would have never expected this, but I’ve switched operating systems for my daily computing. Software for web development sucks a lot less on a Unix-like platform like OS X, even though it’s not perfect. Switching OSs has lead to my decreased dependence on Microsoft’s software. Now, I don’t have a copy of Word on the computer I use most often.

What about those early websites I created or used? Realm of Aaxiler is dead. Scrawlpoint still exists, but has been defunct for ages. It limps along until I decide to eventually pull the plug. This blows my mind: even my own sites didn’t survive the test of time. I lost everything on Writing.com. At some point, Xanga wiped out a lot of content. Fortunately, I was able to export my stuff before that happened. Not a good track record with these sites, right?

And the more recent pieces of software? Google Wave was technically interesting, but died out pretty quickly. Google shuttered it years ago now, taking all the content with it. I had OneNote installed on my Windows machine, but that didn’t seem a good option after switching operating systems.

I still use Evernote and WordPress. Evernote is for quickly capturing ideas and notes. I publish my finished pieces to WordPress. Another system I’ve thrown into the mix this year is everything, which is where I flesh out and edit my writings before publishing them.

The Hard Lesson

My main takeaway from using all this software is that letting someone else control or host my content is a poor policy for making it accessible later. Distilling this a bit more:

Software evolves at breakneck speeds and all else is forgotten.

My second takeaway was the slow realization that my creative works are split across many pieces of software and data formats. Each of the applications and formats is a liability. This concerns me, and I don’t like the risk it brings.

One might argue that the content isn’t valuable if I haven’t touched it in so long. And if it were important, wouldn’t I keep it up to date with new formats and new software? Except, this has no analog on paper. I can write on whatever paper I want, and still be able to read it in 50 years. I don’t have to transcribe it every decade or two so that it remains accessible.

Sure, I don’t need to open writings I created 15 years ago, but I like the idea of it. It’s nice to come across pieces later in life, and feel nostalgia for your past. I want a breadcrumb trail of my works to look back on and see what things were like then. There’s also a chance for serendipitous moments when I reconnect with ideas from earlier days.

These lessons are my main motivators to preserve my older writings. I know, I know… I’m a geek. Good thing I’m an adult, and can spend my free time however I like.

The Right Medium

Earlier, I mentioned that my writings are trapped in graveyards, and the preservation I do now should be aimed at surviving technological upheavals. How can I accomplish this?

Choosing the right medium is paramount. What program, service, or format do I use? This is tricky. I want to access the writings today, as well as at some point in the unknown and distant future. Software has the pernicious tendency to rot, but file formats rot as well.

Any format might seem like a good idea, at least initially. Only later would that choice prove dangerous, when it’s no longer possible to open a file. Programs must support reading, displaying, and writing the format. When the format loses enough popularity, software updates will remove support for it. Even the most prevalent formats will fall out of style and be replaced by newer ones.

Software is generally a form of vendor lock-in. Companies invent their own formats, APIs, and syncing mechanisms. Along with being proprietary, they’re often not interoperable. You place your data in their vault, and there it shall forever stay. Graveyards growing larger by the day.

Each program also has limitations and a lifespan. It won’t be around forever, even if it’s hard to imagine now what would take its place. Ten years ago, I would have expected to still use Windows, and to have Word installed on the computer I use most. That’s not the case, though.

Evaluating Formats

I’ve used various formats, and have writings stored in each of them. My goal is to choose a single one for long term use. This way, I could confidently preserve my existing digital works, and have a solid foundation for backing up other written pieces.

Realistically, choosing just one format itself is risky. I should build redundancy into my process. If my content is stored in multiple formats, it has a better chance of surviving. Picking one format as the source, and then another format or two as fallback would be smart.

To choose, I need to evaluate the options and see what software and formats are suited to this archival purpose. I guess I need some ground rules too. These will allow me to compare formats, and find what is most sensible for my needs.

The format must be:

There’ll be personal preference that gets mixed in here too.

How About Services?

I’m hesitant to heavily invest in new services but if something has the right qualities, I’d consider it. If I consider a service, there are some additional rules.

  • I must be able to export my data.
  • The data export must be in a common format.
  • I must not rely solely on the service’s continued existence.

The third bullet point is most important. I won’t put all my eggs into one basket. All services have uncertainty about how long they’ll be around. That’s when vendor lock-in really comes to bite me in the ass. If I rely on one too heavily, and they shut down, my workflow could be impossible to continue. This is an enormous hazard over the long term.

Let’s Go!

Now we’re onto examining the options and deciding whether they’ll work. If not, I’ll give reasons as to why they get ruled out.

What About Word?

Microsoft doesn’t support the .wps format any more. Eventually, I’m sure the open source software that supports it will cease to. This format is ruled out immediately.

Many of the files I’m concerned with preserving are .doc. Does it make sense use it as the format for the rest of my writings?

The .doc format is binary, which means I need special software to able to work with those documents. It’s not plain text, so I cannot edit it with a basic text editor. The format is also largely proprietary, and Microsoft controls its destiny. Microsoft has made the specification available since 2008, but they still don’t describe all its features. People have reverse-engineered the format, to create other software that works with it, but this is not a format that meets my criteria above.

Without software which knows how to read .doc, my content is inaccessible. Microsoft has created other formats to take its place. In another 10-15 years, it might not be possible to open these files. Or, if it is possible, it could be quite the hassle. Similar to what’s happened with .wps.

So what about something like the .docx format? It’s newer, and has been made into a standard. The latest versions of Word default to this format. Quite a few programs support it, and it’d likely be straight-forward to upgrade my .doc files to .docx. Since it’s backed by XML, the format is, in a way, plain text. But that XML is then zipped, which makes it binary. The advantage is that, even without Word, I ought to be able to extract the content.

I’m not excited by the prospect of tying myself to an XML-based format, even if it is plain text. XML isn’t easy to write in, without software that hides the XML from you, like Word. And using Word would makes me reliant on a proprietary program bound to a single company.

As I mentioned, I don’t use Word much these days. Why bother trying to get myself back into using it? It costs decent money to use and upgrade. It also feels awfully heavy-weight for the writing I do. Sure, there are open-source alternatives that support .docx, but I’ll pass. There are other options left to explore.

What About PDF?

When Adobe’s document-presentation format first came out, there wasn’t an easy or free way to create a .pdf file for my poetry. It was also a proprietary, closed format. Adobe made it an open standard in 2008. Since that time, there’s been an explosion in the amount of software that works with PDFs. For example, it’s easy today to print a PDF from a web page.

Since it is an open standard with a lot of support, .pdf looks like a good option. Except PDF is a weird format. It’s easy to create, but hard to edit. It is well-suited to viewing works, like a hard copy would be. But how do you easily edit a PDF document? That’s a rabbit hole not even I want to peek in.

Since it’s easy to create PDFs from other formats, I will consider it as one of my fallback options. I can use it to back up the final product, since it’s good at preserving the presentation. But I do not want to author works with it, or use it as my system of record.

What About Evernote?

An obvious candidate for authoring and backing up my writings is Evernote. I’ve used it constantly over the last several years. I even pay for the Premium version. It’s particularly adept at taking notes on multiple devices. It has plenty of space, as well as note history. It’s software that’s made for this stuff.

But Evernote uses proprietary software and formats to store the data. My experiences with it leave me wary of depending on it heavily. Some are already calling Evernote the first dead unicorn. I don’t know if that’s true, but I’m sure Evernote as a company has a limited lifespan. The service is the software, and it won’t be possible to use the Evernote software once the company shuts down. That rules it out as a format or writing preserve.

What About WordPress?

WordPress is another application I’ve used for a long time. It’s great at hosting content. The content is mine, and I can export it easily.

WordPress does, however, store that data in a database. The database makes it easy to dynamically build a site, but it’s not a plain text format I can open up with any text editor. Then, the nature of it being a web application means I need a browser and Internet connection to use it. Those are dependencies I don’t really want.

If hackers gained access to my site, I could lose everything. Fortunately, WordPress has plugins to automate data backups to other services like Dropbox. This helps mitigate some of that risk, but it’s just not designed for the kind of use I want.

I plan to continue hosting my blog on WordPress, especially since I’ve automated publishing my finished pieces. But I still have to find something else for preserving and backing up my writing collection.

What’s Left?

I’ve covered what formats and programs and services I won’t use, for various reasons. But what’s left? What can I use? I suppose I should tell you that this post hasn’t been me walking myself toward a decision. It’s been me verbalizing decisions I’ve already made. Trying to understand myself a bit more, and provide insight to anyone else interested. I’ve already settled on a format to use, and have been successfully using it for a majority of this year.

My format is plain, ol’ text files. For styling, I use the Markdown syntax. To indicate this, the files have a .md extension. Markdown draws inspiration from how plain text emails have been formatted over the years. There are conventions on how to indicate things like emphasis, links, quotes, code blocks, and headers. Best of all, I only use regular characters to mark up my content, and these characters are easy to type on any keyboard. This makes it incredibly easy to start and continue using.

Markdown is also a program which converts this plain text syntax to HTML. I’m currently writing this post in Markdown. I’ll later run it through some code that converts it to the HTML you are now reading on this blog.

How long has Markdown been around? Since 2004. That’s not very long, in the grand scheme of things. But it has a lot of pros. It is plain text, well-supported, open, very easy to write in, platform agnostic, easy to back up, easy to version control, and I can create these files on my computer, without needing an Internet connection. Markdown also has a boatload of support on the web.

One downside may be that there are quite a few different implementations of the HTML conversion. The original syntax is ambiguous in certain cases, so programs differ in subtle ways. There are other things the original Markdown syntax doesn’t cover, and extensions of the syntax have popped up, which help to make it more powerful and comprehensive. These extensions might conflict with one another, though.

The awesome thing about Markdown is that even if the software to convert my documents to HTML disappeared, the text would still be usable. Any text editor can open these plain text files. And the syntax itself is a good indication of what the desired styling is. This makes me confident that Markdown can stand the test of time.

Since Markdown (and its various flavors) have a lot of traction, there exist programs to generate HTML pages and PDF documents from Markdown-styled text. That’s a big win. Creating those fallback documents containing the content and style is suddenly much easier.

Another huge benefit is that I’m not tied in to any proprietary pieces of software, or services which could shut down, and the syntax is free to use. I don’t need any special software to use it.

To write my pieces, I’m quite fond of using the free and open source vim. I’ve used it for years to write code, and it feels natural to extend it to my prose as well.

What About Plain HTML and CSS?

Markdown converts to HTML, and that page can then be styled using CSS. Why not cut out the Markdown middleman and go straight to HTML and CSS? They’re also open formats, plain text, platform agnostic, and all the rest.

HTML and CSS are even more widely supported than Markdown. Think of how many billions of web pages use HTML and CSS. They are technologies that will have a long tail. You can even view the very first web page ever created, and it renders perfectly in modern browsers. That’s incredible, considering it was first created in 1991.

I could do this, certainly. But typing the HTML while I write would feel clunky. I’m not trying to write a web site as I flesh out a blog post. I’m trying to work on the words themselves. Markdown seems like a nice compromise. I can work on the content, easily add some markup as I go along, and then convert it to HTML when I’m ready to publish. At that point, I can consider other details of styling, if the document requires that attention.

The HTML output from Markdown is itself another fantastic fallback document. It essentially comes along for free, thanks to the qualities of Markdown.

The Backup Strategy

Additionally, I need a process for backing up the writings. I want them safe, even if it’s years before I access them again. A successful backup strategy would allow me to lose one copy of a writing, and still be able to recover that document. This way, an accident, user error, or hardware failure wouldn’t mean a piece disappears forever.

In reality, the best backup strategy is actually multiple, redundant, backup strategies. Having many copies in many locations is even better for covering my ass.

My hesitation about services dissipates when I don’t rely on them too heavily; when a service isn’t a single point of failure. If I use a service as a backup destination, I feel better about it, because it’s one of several.

I love services like CrashPlan and Dropbox, because they’re transparent. I can use both at the same time, seamlessly. They back up files on my hard drive, which is about as unobtrusive as you can get. Backup strategies are fantastic when you can layer or combine them. For instance, it’s easy to throw in another option like Apple’s Time Machine, without changing anything about the others. They just work like normal.

I could even consider Evernote as a backup site. Evernote documents are HTML, so I should be able to take the HTML output from my Markdown piece, and stick it into a note. I’ll have to look into automating this process in the future.

Karla suggests I should print out hard copies and store them in a safe deposit box. Storing backups away from the home is a good way to keep a natural disaster, burglary, or fire from wiping out my data, but I wonder what the costs and effort associated with that are. Like she’s said, it’s not as elegant as the other options, but it is certainly interesting as a last resort. It could be the fallback to all my other fallbacks.

Version Control

For version control, I use git. It’s been around for 10 years, and has strong support around the globe. I’m familiar with git because I use it to version control software, and it turns out to be a great option for putting plain text writings under version control too.

This allows me to keep a history of changes I make to pieces. And git gives me the ability to push my writing repository to multiple places, like BitBucket or GitHub. Each of these places has a full copy of the history, which means they can serve as backups too. This is yet another way to back up my content, along with keeping track of my changes.

Conclusion

I’m using plain text, Markdown files to store my writings. I write using vim. I can convert them to HTML and PDF pretty easily, as fallback formats. I store the Markdown writings in a git repository, and these all get backed up to BitBucket, Time Machine, Dropbox, and CrashPlan. This feels like a robust way to back up my digital writings. It’s something that also ought to scale as I back up more writings in the future.

It’s been helpful to think through these decisions, and I hope this might help someone else. I’d love to hear your thoughts, if you’ve considered these topics at all. I’m sure there are things I haven’t considered, or things I’ll only learn down the road.

I’ve got other goals for writing-focused, web-based software, which I hope to explore. Writing this year helps give me more experience to use when I undertake those software projects in the future.

Thanks for reading, and here’s to writing!

What Drives the Worlds of Fiction?

The Lord of the Rings has a vast world, but an even richer history. This story and the related history are people-driven. The world is subservient to the actions of the beings within that world. What do I mean by this? It’s apparent when we look at Middle-earth itself. The areas of the world take on the characteristics of those dwelling there.

Greenwood the Great became Mirkwood after the Neromancer took up residence. Rivendell is rich, deep, and secure, just like the elves who created it. Lothlorien is golden, dreamlike, and beautiful, as are Galadrial and Celeborn. The Shire is quiet, peaceful, and full of good food, like the Hobbits. Mordor is wretched, ruined, and powerful, because Sauron imparts his will upon the land. The Old Forest is odd, just like its master, Tom Bombadil.

The stories of Middle-earth reflect this as well. Calamities are brought about, most often, by beings. Beings have more a chaotic, unpredictable, awe-inspiring role than does nature. The planet bends to the forces of the characters. This is interesting in that it’s nothing like the world in which we live.

We project the fortitude and resilience of the American people on the Rockie Mountains, but we surely did not make them rise up. Hurricanes, avalanches, and earthquakes happen regardless of the personalities of those living in the area. In fact, for us, the lay of the land can influence the people. Storms cause damage which impacts citizens. Forest fires cause us to evacuate our homes. Mountain ranges feel comforting. Does nature help shape who we are?

Let’s consider another fictional world – that of Dune. It contains humanity and political machinations, but it’s all predicated on a harsh and powerful world. The people bend to the desert; the economy is driven by the spice, “the most important and valuable substance in the universe”. Civilization is built on the back of the ecology of the physical world, and it deals every day with still suits, worms, and sand. This is a reversal of the approach taken in Middle-earth. It models more closely our actual world.

Is it a view from bygone days that all inhabitants of an area are the same? A sort of stereotype of a population. We’re smart, brave, and persistent. We’re American. We conquer the challenges we face. Outsiders are fearful, cowardly, and wrong. They’re foreigners. Their personality brings about the disasters they face. This mindset has no basis in reality. People of all sorts live all over. The brave and the cowardly live side by side in every country.

I can see why stories like those of Tolkien are popular. They give us a taste of what it’d be like to live in a fantastical setting where personality determines nature. It inflates our role in the world, and makes us believe we have more control. We anthropomorphize our surroundings based on our qualities. It’s a change from our everyday experience where we’re subject to the vagaries of Mother Nature.

Are there names for these approaches to portraying and fictionalizing the world? People-as-servant versus World-as-servant?

This idea recently came to me as I thought about potential stories. I’m influenced by things like Pikes Peak, which I lived near for four years. The Waldo Canyon fire also left an impression. The groups I imagine are impacted by the world and that drives action just as much as conflict with other groups. It’s fun to compare my thoughts to approaches other authors have taken, especially those who have created fictional works so popular. I’d love to hear your thoughts on this subject.

All Software Rots

Software is an interesting creature. It’s easy to imagine that a piece of software is, at some point, completed, and goes on to live and run for eternity. The group who wrote it retires happy and comfortable, based on the success of that software.

Except, this is not reality. Over time, software degrades, runs more slowly, or doesn’t run at all. Putting software out there and expecting it to be useful in the future means we’ll have to work at it. It’s a consideration I’ve only recently realized. What causes this?

A technology stack involves a lot of moving parts. Hardware, firmware, drivers, operating systems, libraries, frameworks, browsers, APIs, and applications are all in this mix. There are other things I’m not even aware, I’m sure. And these stacks support the existence, operation, and functionality of an application.

Each of these components has a purpose, design limitations, bugs, shelf life, and life cycles. Security vulnerabilities, performance issues, and annoying bugs are likely present too, perhaps to be discovered decades down the line. Software written to run on systems in the 1960s have different stacks than those written in the 90s than those written today than those written in a few years.

When one piece of the stack goes extinct, the change ripples upward. Have you tried using games or other applications from a decade ago? It’s not easy. Large programs with many dependencies are at the most risk. More pieces can rot out from under it, ruining the beautiful cathedral which many human-hours built.

Services are no exception to this rot. Take Heroku for an example. Developers can deploy their web application without having to worry about the nitty gritty of configuring and managing a web server. The tradeoff is buying into the platform Heroku provides, and their company. These are also vulnerable to rot.

Note: I use the free tier of Heroku, so I can’t really complain. It just happens to be the Platform-as-a-Service I’m most familiar with.

Even if you don’t make any changes to the functionality of a web application, you’ll have to work to maintain it. It won’t run for 10 years on its own. These platforms will upgrade their software, practices, and capabilities. Tech stacks will fall as others rise.

I use the word ‘rot’, which has certain connotations. But it’s similar to the rotting of a house. Termites just do their thing, and it results in the foundation of a house changing in ways that are structurally problematic. It’s not a moral judgment; it’s neither good nor bad. Similarly, a tech stack rotting is neither good nor bad. Developers, software, and hardware just do their thing, and it results in the foundation of an application changing in ways that are structurally problematic.

Dynamic sites have more risk of rot than do static sites. The dynamic part means there’s software, production code, and databases. Security holes, bugs, and cosmic rays will have more chance of impact. There are more moving parts that need tending to.

Sites serving up static assets like HTML, CSS, and Javascript have a better chance to survive into the future. There are fewer moving parts. Static website generators are quite popular today, for many reasons, including their speed, costs, and ease of use. They’re curious to me because the output has the potential of a long lifespan.

The generator itself can be subject to rot, but that doesn’t impact the site you’ve already created. You can switch out the toolchain, and, as long as the site created is the same, the code is transparent.

I’ve currently got The Mechanism Collection hosted as a static site on GitHub Pages, for free. I get to piggy back on GitHub’s infrastructure, but I could take the static pages and host them elsewhere, if needed. Perhaps the Pages infrastructure will rot in the future.

Static sites are something I’ll explore more going forward. Especially for hosting writings, like this blog. I don’t author my content in WordPress anymore. The comments here don’t get used much, because the conversation happens on social media, where I share the links. There are 3rd party services for comments, although I’d be wary of using one. The blog itself could be static, and visitors likely wouldn’t even notice a downside. The benefit of speed would be noticed and appreciated, however. Keeping URLs to posts the same would make the change even more transparent.

Sentient machines are a lofty goal, but what happens when there’s a zero day that turns that ‘being’ into an intelligent botnet? Rot is job security for software engineers. We have the opportunity to create lasting works of code. We’ve got some work ahead of us though.