I was reminiscing to a co-worker about my time in Japan, and told him a story
that I’m not sure I’ve ever told anyone else. So, why not regale the internet
with pointless personal anecdotes?
For those who don’t know, I taught ESL in Japan from the summer of 2004-2005.
Somehow I had ended
up connecting with the owner of a private school that was located in
Utsunomiya.
Definitely something of a backwater locale, even though it’s only 60 miles
away from Tokyo. Most of my teaching was in the main office that was close to
the center of the city, but I also had to regularly commute to a satellite
office that was in nearby Tochigi
(bizarrely, on a rail line operated by a department store). The school I worked
for, being a small family-run business, had all ages of students: I sometimes
“taught” preschool kids, and my oldest student was a guy in his 80s (whenever
he missed a class I worried that he had died).
One of my students at this satellite office was a junior high student named
Moe (pronounced moh-ay). She was actually one of my better students; since
she was more fluent in English, our lessons were more high-level, and therefore
less boring for me. Towards the end of the 2005 school year, my Japanese co-worker
and I had finished a lesson with Moe, when she invited us both to a concert that
her school band was giving. While my co-worker demurred, I thought, “Why not?”
and told her I’d go (toward the end of my year-long contract, I’d determined
that I was not going to stay in Japan, so attempted to have a “try anything”
attitude during my remaining time).
The concert was on a Sunday, and the day before I was working in the Tochigi
office as usual (yes, I had to work on Saturdays,
and got Sunday/Monday off). Imagine my bemusement/concern when, at the end of the
day, my co-worker suddenly pulls out a giant bouquet of flowers. “Here, you can
give these to Moe.” she blathered. I worried about what people might think of a
20-something foreigner giving flowers to a young female student, but felt
obligated at the same time. Cursing her under my breath, I took the bouquet,
trying not to think about what I was going to do with it.
The next day, for the first time ever, I took the train to Tochigi for non-work-related
purposes, and walked to the school. It felt pretty weird to be commuting on my
day off. I ended up stuffing those damn flowers into my backpack, and figured
I’d worry about them later. As I walked into the school auditorium’s foyer, I
could almost hear people’s necks snapping as they turned to look at me. Of course,
I was the only non-Japanese person there. And I don’t know why I didn’t think of
it before, but of course it was an all girls school. The awkwardness of my
presence had just increased exponentially. I brazened it out and found an
inconspicuous seat near the back.
From what I remember, the concert itself was actually enjoyable. I didn’t go in
expecting too much (remembering my own school-age band concerts), but as you
might imagine, the Japanese always seem to take things to the next level. Not only was
the music played impeccably, they also threw in some choreographed movement with
their instruments.
After the thing ended, the students dispersed into the audience to receive
congratulations from their various family members, and I had to deliver that
albatross-like bouquet. Fortunately, I was able to hand it over pretty
inconspicuously, thanks to the crowd.
After the meet ‘n greet, the students were called back up to the stage for a
group photo. Imagine my chagrin when the photographer, seeing a white guy hanging
around, yelled, “Who’s he with?” When it became known that I was Moe’s English
teacher, he told me to get up on the stage for a photo, a suggestion that was
enthusiastically received by the girls themselves. “Jeez, you’re tall!” I
remember one remarking.
So, somewhere in Tochigi, there’s a photo of an all-girls junior high jazz band
with a nerdy guy standing in the middle of the group, trying desperately not to
look as awkward as he feels. Good times 😅.
I seem to continually have dalliances with the Android world. Last year I bought
(then sold) a Sony Xperia Z3, then later purchased/returned an LG Nexus 5X.
I always seem to end up going back to my trusty iPhone 5, though. There are
just enough dealbreakers with any Android phone that make them tough to live with
as a daily driver.
Earlier this spring, I was revamping a few of my iOS apps, and figured that
since they’re written using JavaScript and published as “apps” in a Cordova
wrapper, I might as well try publishing on Google Play as well. I’d returned my
Nexus 5X long ago, so started trolling Swappa for a used
Android phone. I’d had my eye on the Xperia Z3 compact for a while: its stylish
design, coupled with a large screen inside a small(-ish) chassis had me itching
to try one out. I grabbed one for a decent price, released my apps
to absolutely zero downloads, and then tried using it as my main phone for a bit.
The thing that I love the best about the Z3 Compact is its screen-to-body ratio.
The whole device is just slightly larger than my iPhone 5, but the screen is
about 33% larger. Watching videos and reading books is much more enjoyable.
Plus, since the overall phone size is manageable, you can use it without
resorting to the “smartphone claw grip of death.” The body of the phone looks
pretty classy as well – I got the black version, which is like the prototypical
black slab that all other smartphones descend from. Unfortunately, Sony never
evolved its industrial design past the iPhone 4 era: the phone is all glass,
save for the plastic sides. I always treated it fairly gingerly, as I was never
sure how hard I could set it down without cracking the back.
Unfortunately, the software front is where most Android phones fall down for me.
While most of Google’s software is on point (Chrome, Gmail), the Android
messaging scene is pretty fragmented. What I want is a Google-endorsed
messenger that piggybacks on top of SMS, similar to Apple’s iMessage. The
app that might have fit that category (Hangouts) is deprecated, and users
are advised to use a standalone SMS app. Rather than double down on Hangouts,
Google instead is promoting two new messengers, Allo and Duo, that you’ll have
to get all your friends to download in order for them to be useful at all. Blah.
At that point I’d rather use Facebook Messenger, which at least most people
have installed.
Sony also tries to include a lot of “value-added” software, which is mostly
worthless. They have all sorts of extra media apps, but without a compelling
syncing solution for my Mac, I didn’t load any music or videos onto the phone’s
internal storage. A few bloatware-type apps (AR Fun, wtf) were immediately
disabled (can’t delete these apps, of course). I could sign up for a “My
Xperia” accout, but it’s not immediately apparent what benefit it would give me,
aside from yet another set of account details to remember (cursory internet
research tells me it’s similar to Apple’s Find My iPhone).
Dispite these annoyances, the phone was a pleasure to use, once I disabled/
uninstalled offending Sony software. Since I mostly rely on very basic
smartphone apps (Maps, Email, Browser, Books), I don’t mind missing out
on the hottest new iOS microtransaction bandit, errrr, game.
However, I ended up getting rid of the phone. Since my wife
has an iPhone, the lack of iMessage is a real killer. She’d try to send me a
video of the kids, and it would be sent as a hyper-compressed MMS. Static
images would fare no better. Rather a minor thing, but messaging is the
core of my smartphone use.
The other unfortunate thing is that the Z3 Compact has no upgrade path. It
shipped with Android 4.4, and has been (slowly) upgraded through Android 6,
but it’s anyone’s guess how much longer it will receive updates. While my
iPhone 5 is nearing the end of its update life (4 years!), I know that I can
buy a new phone with updated internals in the exact same form factor, and
have it be supported for another 3+ years. The Z3 Compact has no obvious
successor. Sony released the Z5 Compact last year, but it uses the problematic
Snapdragon 810 processor, and the US version has its fingerprint sensor
disabled(!). The Z5 Compact’s design is also a bit more stodgy. And the
Z5 series is the last of the Z’s – Sony’s recent crop of X-series phones
are about as forgettable as they come. I don’t want to get too attached to
a dying phone.
I guess I’ll appreciate the Z3 Compact for what it was at the time, and
hope that one day Apple will release an iPhone 5-sized device with an iPhone
6-sized screen.
Last year, when I ditched my managed hosting, I converted my blog to use Jekyll,
a static site generator. At the time I was pretty overwhelmed with all the tasks
necessary to move my digital life, and didn’t want to throw “configure PHP/MySQL”
onto the pile. Also, much to my chagrin, when I looked over my Wordpress installation
prior to exporting its content, I noticed a bunch of suspicious-looking files
that could only have been created by script kiddies exploiting Wordpress vulnerabilities.
I thought I had been pretty concientious in keeping my instance up to date, but
apparently not. So a static site it was.
One of the downsides to a static site is that… it’s static. No comments or
any other form of interactivity. At first I tried hooking up Disqus, which
I had actually also used in my Wordpress blog, rather than the native comments.
Problem is, I dislike Disqus. As with any free service, if you aren’t paying,
then you are the product. Managing a Disqus account is annoying, and it’s yet
another 3rd party service that gets to track you around the web. So, during a
bit of downtime, I created my own basic blog comments app.
It’s a Ruby app written with Sinatra that has two routes: GET and POST. Comments
are stored in a SQLite database (I’m not anticipating heavy traffic). Include the
client-side script wherever you want your comments to appear in a post. It uses
reCAPTCHA for abuse prevention.
I learned a few things doing this project, including how to use nginx as a reverse
proxy (only used Apache before), create an Upstart init script (sigh), and
deploy a basic Ruby app on a VPS. It’s amazing how much of this stuff you don’t
have to do when working at a company with sysadmin folks XD.
It should be fairly easy for anyone to take this code, made a few minor modifications,
and run it for their own blog. I would also be happy to answer any questions
about getting it up and running… just leave me a comment!
Work on a web app long enough, and you’ll come across some sort of task that needs
to be accomplished by your app at regular intervals. Some examples might include
pruning inactive user accounts, sending reminder emails, or running custom analytics.
My first introduction to these sorts of tasks was at a previous job. A Rails
app needed to run a task at regular intervals, and used a gem called Clockwork.
At first glance, this seems to be a decent solution to the problem – a lightweight
Ruby daemon which triggers your code at regular intervals, using a nice DSL.
However, after using it for a while, I decided it perhaps wasn’t the perfect tool.
First of all, it adds a dependency to your app, and you need to take the time to
learn the API and how to use it. You also need to monitor the Clockwork process
to ensure that it restarts after a crash. You also need to ensure that the Clockwork
process runs on a single app server; otherwise your job will be run multiple times.
Plus, it turns out that timezones are hard.
When a recent requirement came up that could be solved by a periodic task, I ruled
Clockwork out, and decided to use some other common tools to accomplish the same goal.
As you may have surmised, I ended up using cron to invoke curl, and hit a
private URL in my app. While I’m sure many folks can point out downsides to such
an approach, it has a lot of appeal to me:
Uses very basic, well-understood software that ships with most (all?) Linux systems.
Hits your webapp, so you don’t have to deal with setting up a DB connection pool, logging, etc.
Cron syntax inscrutable to you? A quick Google search
should clear it up nicely. Don’t want to expose a sensitive or resource-intensive
task via a public URL? Use basic auth to secure it, returning a 404 if the auth
is missing. As a bonus, cron will use your systems time, so you don’t have to worry
about the scheduled job missing a beat during DST (yes, this did in fact happen to me).
For a recent project, I was tasked with creating a large number of user accounts
on various 3rd party websites. Specifically, Spotify, Last.fm, and Basis. I started
off by doing the ol’ copypasta, but after about the third Spotify account, I realized
that I was going to go brain dead before finishing them all. Then I remembered
Selenium. I had my list of proposed usernames, email
addresses, and passwords – why couldn’t I just feed those into an automated browser
instance?
Well, the answer is, of course, that I could – with varying degrees of success.
For most account-based websites, the general steps you’ll take to create a bunch
of accounts is as follows:
Go to signup page
Enter user data
Submit form
Log out
Spotify was the easiest of the three to automate. Their signup form is very basic,
with (surprisingly) no CAPTCHA or email validation. The only problem is that
when you successfully submit the form, the resulting page automatically prompts
a desktop client download. I wasn’t able to quickly find a way to dismiss that,
so simply paused my script so that I could manually click “cancel.” Much easier
than filling out the form.
Last.fm was a bit tricker, as they include a reCAPTCHA widget in their form.
While modern versions of reCAPTCHA look like just a simple checkbox, there is
obviously some complex logic going on behind the scenes. As a human, you click
their “I am not a robot” checkbox and you’re done. When a script fills out
the form and clicks the checkbox element, the checkbox is checked, but an
image recognition CAPTCHA is presented. Rather than try to figure out the logic
behind triggering the additional level of protection, I simply stopped my script
after loading the challenge, and manually clicked through. It was kind of fun,
actually, to see the different queries and how fast I could complete them.
I suspect the CAPTCHA is triggered by the absense of human-like activities
(i.e. normal JavaScript events that would be triggered by a human, like mouseover),
but didn’t feel like figuring it out, because I’m lazy.
require "selenium-webdriver"
require "pry"
driver = Selenium::WebDriver.for :firefox
accounts = [
# account info redacted...
]
accounts.each do |account|
driver.navigate.to "https://secure.last.fm/join"
driver.find_element(:name, 'userName').send_keys(account[:username])
driver.find_element(:name, 'email').send_keys(account[:email])
driver.find_element(:name, 'password').send_keys(account[:password])
driver.find_element(:name, 'passwordConf').send_keys(account[:password])
driver.find_element(:id, 'id_terms').click
captcha_iframe = driver.find_element(:css, 'iframe[title="recaptcha widget"]')
driver.switch_to.frame(captcha_iframe)
captcha_checkbox = driver.find_element(:id, 'recaptcha-anchor')
driver.action.move_to(captcha_checkbox).click(captcha_checkbox).perform
# switch back to the main document
driver.switch_to.default_content
# Wait until Control+D from user to continue
binding.pry
end
Basis was the most annoying form to fill out. It’s some sort of terrible SPA,
so you have to wait until the DOM content loads before filling in any data.
It also has some sort of weird logic that requires each input to be tabbed into,
or else it thinks the POST fails when the form is submitted (it’s not, actually).
You can also tell it’s some horrible JavaScript form because some input elements
don’t actually have name attributes, which also makes them terribly annoying
to find/fill out. I ended up enumerating over all the input elements, then trying
to differentiate based on a data-bound-to property. The problem was that multiple
elements had the same data-bound-to value, so I had to ignore any subsequent
elements after the first. This form also asks for a lot more data (name, height, weight, etc.)
that I had to generate. That’s why I love Ruby’s Array#sample method.
require "selenium-webdriver"
require "pry"
driver = Selenium::WebDriver.for :firefox
accounts = [
# account info redacted...
]
accounts.each do |account|
driver.navigate.to "https://app.mybasis.com/#register"
# It's a stupid Javascript app, so wait until the DOM elements load
wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
wait.until { driver.find_element(:name, 'email') }
# Email
driver.find_element(:name, 'email').send_keys(account[:email])
driver.find_element(:name, 'confirm_email').send_keys(account[:email])
# Password
driver.find_element(:name, 'password').send_keys(account[:password])
driver.find_element(:name, 'confirm_password').send_keys(account[:password])
# Birthday
birthday = "#{(1..12).to_a.sample.to_s.rjust(2, "0")}/#{(1..28).to_a.sample.to_s.rjust(2, "0")}/#{(1970..2000).to_a.sample}"
driver.find_element(:name, 'anatomy.dob').send_keys(birthday)
# Gender
driver.find_element(:name, 'gender').click
# Timezone
driver.find_element(:name, 'settings.timezone').send_keys("(GMT-04:00) New York")
# Find elements that don't have any sort of queryable attribute
height_counter = 0
weight_counter = 0
elements = driver.find_elements(:css, 'input')
elements.each do |elem|
case elem.attribute('data-bound-to')
when 'n:profile.first_name'
elem.send_keys(%w(James John Robert Michael William Mary Patricia Linda Barbera Elizabeth).sample)
when 'n:profile.last_name'
elem.send_keys(%w(Smith Doe Brown Jones Johnson Miller Davis Wilson Anderson Taylor).sample)
when 'n:anatomy.height'
if height_counter == 2
next
elsif height_counter == 1
elem.send_keys((0..11).to_a.sample)
else
elem.send_keys((4..6).to_a.sample)
end
height_counter += 1
when 'n:anatomy.weight'
if weight_counter == 1
next
else
elem.send_keys((110..180).to_a.sample)
end
weight_counter += 1
end
end
# Manually submit form, then log out
binding.pry
end
I’ll probably never have to do anything similar in the near future, but hopefully
these examples will help you out if you have a similar (mind-numbing) task in your future.