Building Custom Ruby From Scratch On Osx 17 February 2018

Building & Using Your Own Ruby

If you want to work on changes or learn more about the internals of the Ruby language, you can alter the source and build your own from scratch. It isn’t that hard or scary, you will learn a bit more about Ruby just by building it. This is a quick start to building Ruby from scratch on OSX, and using it to run your local apps.

Get the Source

Use git or SVN to pull the source repo.

  • I like git, so I forked the Ruby project.
  • Then clone it: git clone [email protected]:danmayer/ruby.git
  • go into the directory and take a look around.

Build Trunk Before Modifying

Before you make any modifications, I recommend you get the current trunk building and test running your apps with it. Then you can create a branch and see the impact of any modifications you would like to try out.

Building Yourself

I recommend using the Ruby-buil instructions below as I continued to hit issues with building from scatch with my own options related to SSL.

If you want to give it a shot thought, run the 4 commands below and it will compile and install Ruby from src on your system.

aclocal
autoconf
bash -c './configure'
make && make install

If you get this error about OpenSSL:

openssl:
	Could not be configured. It will not be installed.
	Check ext/openssl/mkmf.log for more details.
*** Fix the problems, then remove these directories and try again if you want.

It is a bit hard to resolve on OS X yourself, but luckily we can just use ruby-build to do the work for us, see below.

Building via Ruby-build

Ruby build will detect and use SSL from homebrew and avoid the broken OSX implementation.

  • install ruby-build: brew install ruby-build
  • copy the current dev trunk target: cp /usr/local/Cellar/ruby-build/20171226/share/ruby-build/2.6.0-dev /usr/local/Cellar/ruby-build/20171226/share/ruby-build/2.6.0-mine
  • edit the file (...ruby-build/2.6.0-mine) to point to your fork: ... install_git "ruby-trunk" "https://github.com/danmayer/ruby.git" ...
  • build from your git to the ~/.rubies/ directory: ruby-build 2.6.0-mine ~/.rubies/ruby-2.6.0mine
    • run from the build dir: /usr/local/Cellar/ruby-build/20171226/share/ruby-build
  • make sure to open a new shell so ChRuby (or RbEnv) find the new ruby.
  • you can now reference the new build in your .ruby-version file in any project

If you want to build from your local git to avoid pushing to a remote branch while testing this is how your build file should end up.

install_package "openssl-1.1.0g" "https://www.openssl.org/source/openssl-1.1.0g.tar.gz#de4d501267da39310905cb6dc8c6121f7a2cad45a7707f76df828fe1b85073af"  mac_openssl --if has_broken_mac_openssl
install_git "coverage_pause" "/Users/danmayer/projects/ruby" "feature/coverage_pause" ldflags_dirs autoconf standard_build standard_install_with_bundled_gems verify_openssl

If all is working as expected you should see this.

ruby-build 2.6.0-coverage ~/.rubies/ruby-2.6.0coverage
ruby-build: use openssl from homebrew
Cloning https://github.com/danmayer/ruby.git...
Installing coverage_pause...
ruby-build: use readline from homebrew
Installed coverage_pause to /Users/danmayer/.rubies/ruby-2.6.0coverage

Playing Nice with Other Rubies?

If you are like most Rubyists you have a number of Rubies installed, beyond the default OSX Ruby, using something like RBenv, RVM, or ChRuby.

  • The default OS X Ruby should be: /usr/bin/ruby
  • If you build from scratch the default target will be installed to: /usr/local/bin
    • This can cause some issues with various Ruby environment managers
    • adjust your path to target this Ruby or your normal one
    • Another reason I recommend building via ruby-build, as it will work easier with most Ruby environment managers
  • Ruby build will put ruby into the ~/.rubies
    • with ChRuby it will automatically pick these up

Then you can reference your new custom ruby in a .ruby-version file at the root project directory, for example:

# .ruby-version
ruby-2.6.0coverage

Running an App with your Custom Ruby

After building your own Ruby and setting the .ruby-version you should be good to go. You can verify you are running your Ruby by adding some simple print statement. Pick a favorite Ruby method and just add a print statement like so…

printf( "hello from my method!\n" );

  • push the change to the branch your referenced in your Ruby build steps
    • there are ways to build locally, but I have just targetted git branches
  • rebuild Ruby: ruby-build 2.6.0-coverage ~/.rubies/ruby-2.6.0coverage
  • enter your project iwth the set ruby-version
  • run ruby -v to make sure it matches expectations
    • if you get chruby: unknown Ruby: ruby-2.6.0coverage either refresh chruby or open a new terminal so it picks up the new build
  • now run things as you normally would…
# in Rakefile
desc "call coverage running"
task :call_coverage_running do
  require 'coverage'
  Coverage.running?
end
rake call_coverage_running
hello from Coverage.running

In the above output, I can see that I am runnin my custom Ruby branch as I had added a print messaged to the Coverage.running? method.

Testing your Custom Ruby

If you are trying to do some real work on Ruby other than exploring, you will want to be able to run the tests on files after you modify them, and add your own tests.

Running Tests

For example to run a specific test file, you can run the below command.

make test-all TESTS=test/coverage/test_coverage.rb
Run options: "--ruby=./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/

# Running tests:

Finished tests in 1.580204s, 12.0238 tests/s, 74.6739 assertions/s.
19 tests, 118 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.6.0dev (2018-01-13 trunk 61811) [x86_64-darwin15]

Running Specs

Below we can see how to run a single specific spec file

make test-spec MSPECOPT=spec/ruby/library/coverage/start_spec.rb
generating x86_64-darwin15-fake.rb
x86_64-darwin15-fake.rb updated
$ /Users/danmayer/projects/ruby/miniruby -I/Users/danmayer/projects/ruby/lib /Users/danmayer/projects/ruby/tool/runruby.rb --archdir=/Users/danmayer/projects/ruby --extout=.ext -- /Users/danmayer/projects/ruby/spec/mspec/bin/mspec-run -B ./spec/default.mspec spec/ruby/library/coverage/start_spec.rb
ruby 2.6.0dev (2018-01-13 trunk 61811) [x86_64-darwin15]
[\ | ==================100%================== | 00:00:00]      0F      0E

Finished in 0.002753 seconds

1 file, 0 examples, 0 expectations, 0 failures, 0 errors, 0 tagged
comments

January Software Links 03 February 2018

Some of my favorite links from January. Also, I will use this month to highlight a group of folks who have been sharing advice and helping me grow for a number of years now. Thanks!

The above folks in a mentoring group recommend good books, bounce ideas off each other, and help folks prep for big talks, events, or projects. Thanks!

Software Development

Data

Ruby

Tech Management

Random

Random Links

Links by wsyperek

comments

December Software Links 31 December 2017

Some of my favorite links from December. If you have any good links I missed pass them my way.

continuing my trend of having something from @davetron5000 each month, and this month I have two ;)

Software Development

Ruby

DevOps

Tech Management

Random

Random Links

Links by wsyperek

comments

On Tech Challenges 18 December 2017

Coding Challenge

The company I am with, Offgrid-Electric, has evolved our hiring process over time to try to ensure we can find a diverse and strong distributed team. There are many interesting processes related to hiring a distributed team, but I will quickly focus on the coding challenge, technical challenge, or whatever folks are calling it these days.

Job Challenge

image by FotografieLink

Offer Challenge Options

This post is kicked off by the thread below.

Folks followed up asking for more details on other challenge options we provide and why, which I will try to breakdown below.

What is the point?

First off why have a coding exercise in person, take-home, or other? The goal is to allow a candidate to demonstrate skills beyond being great to talk with during interviews. That they can back up the conversation with problem-solving skills that match the team’s expectations. There are far too many posts and stories joking about how few developers can code, which I will skip over, but often a team wants a way to help ensure that a candidate has skills that will make them successful if they join. At OGE we try to allow for the widest possible options to prove out that a candidate would be technically qualified or the position.

Tech Challenges Options

We offered a few standard options and have added more and evolved our challenges over time.

Take Home Challenge: We have someone on our team develop a challenge for any job description we are hiring for. The challenge is prepared before we even post a job opening. In general, we have someone on the team develop the challenge and complete it themselves to ensure they fully understand the challenge and have a good estimate of the time required to complete it (generally 2-4 hrs). This falls somewhere into the category of building a small application that does X, Y, and Z given this sample data set. Given folks likely have a current job we give plenty of time to work on the challenge and include instructions for how to submit the challenge and things we expect to see.

Pair Programming: We had some folks talk to us that enjoy pair programming and find it a good way to understand the way our current team works and thinks. We offer to pair on the take-home challenge and work on it with the candidate at a scheduled time, or to work on some of the code we are currently working on, or to pair on some code that the candidate has recently been working on. Some folks are very comfortable with this kind of exercise while others get very nervous pairing with someone they are just getting to know. Some folks have paired a lot in the past, or it might be a fairly new experience for them. Unless your team practices 100% pair programming, I believe you should offer some alternatives to just pairing. I like having a take-home challenge that can also be offered as a pairing exercise.

Open Source Work: This is a great option for folks that already have a lot of publicly available code they are proud of and showcases their skills. There is no reason to make someone spend extra hours of their time when you could have them walk you through code they are passionate about and have already been working on. I have enjoyed learning about some candidates projects even if it wasn’t a great fit and the enthusiasm for something they have recently worked on in their own time shows through. It is important to have other options beyond just open source. While this is a great alternative, it can’t be the primary or expected way to check off the technical challenge. You can find more detailed reasoning about this on various diversity hiring posts, but not everyone has the same privilege to spend extra time outside of work on open source, some jobs don’t allow it, it isn’t everyone’s cup of tea.

Technical Questioning: While I prefer to do something with code like the options listed above, sometimes that isn’t the best fit for whatever reason. In these cases, we have done some technical white boarding type interviews over video chat. In my interview, at OGE, I walked through how to split out a monolith application into smaller component applications and discussed the kind of DevOps you need to support such transitions. This format has also worked well for QA interviews. I try to avoid trick questions / brain teasers. Asking data modeling questions tends to work pretty well over a whiteboard for an example of good technical questions.

Choose your own Adventure

Taking this even further, given our only goal with this part of the interview process is to come away with some confidence that the person is technically capable in a set of skills… I often ask the candidate after presenting our standard challenge options listed above if that have any other way they would like to prove out technical skills. I will list some things candidates have suggested or done before.

Technical Writing / Educational Materials: We have had some folks apply that have created online materials for courses. Sharing video series that teach programming concepts, blog posts detailing how to use some technology, or videos of talks given at a conference covering technically relevant material.

School Projects / Papers: If you are talking to someone recently coming out of university they might have some great projects they could share with you. If it is a group project just get into specifics about what pieces they directly contributed. We recently had a candidate submit their Ph.D. thesis as a way to confirm their technical skill.

Code Reading / Walkthrough: A few candidates that didn’t have open source code but as consultants had learned to quickly look at projects and distill how they worked. In this case, they were also curious to see how our systems worked. After some discussion, we walked through sections of our companies source code with the candidate reading the code and explaining what it did and the recommendations they would have to improve that section of code. While this is a bit more challenging on the interviewer’s side, it can give candidates a great picture of the kind of code and the quality of the project they will be getting involved in. If the person can quickly jump in and absorb parts of the project’s domain and explain it back to you, it can give you the confidence you need to move forward.

Comfortable Candidates Can Share Their Skills

In the end, it comes down to just making sure you set a candidate up to show you their best work. You don’t want to setup a situation that will have a great candidate so nervous they don’t show what they are capable of. Being a globally distributed team, in the end, I don’t expect all our team to work the same way, so I wouldn’t expect all candidates to succeed in the same style of challenges. You can always find a developer that loves to pair or hates it, or gets very nervous working out a problem on a whiteboard or could lecture in front of a whiteboard for hours on end. All of that is fine, let em shine.

comments

November Software Links 02 December 2017

Some of my favorite links from November. If you have any good links I missed pass them my way.

Software Development


intentionally and proactively providing an extra-normal level of support and manual help to users in order to learn about the barriers people are hitting

service and support driven development

Data

Ruby

Devops

Tech Management

Random

Random Links

Links by wsyperek

comments

Active Record Database Documentation 12 November 2017

Database Documentation

Documentation Folders CC image from Pixabay

Active Record Database Documentation

This post will cover how to use the Active Record feature to add comments to your tables, columns, and indexes. This feature makes it easy to keep your database documentation up to date as you can add descriptions at the same time as you add or update tables and columns.

Documentation Embedded with Change Process

Why would we want to use Rails to build our database documentation?

I believe documentation close to code and embedded in the code change process has a better chance of staying up to date and relevant.

I also think we want to add human context to our information in same tools we build our database. We want to do this in source code that is trackable for the same reason we run database migrations in Rails opposed to just having a DBA make schema changes outside of our application code change process.

When we embed our database documentation in our standard code change process we easily get many advantages.

  • See DB comments change over time because they are part of Git
  • Search in code editor tools (and github)
  • Documentation can be reviewed as part of PRs by a data team, analysts, or other folks who might be the target documentation audience

By having the documentation embedded in the database directly, other values can be unlocked.

  • The documentation is embedded in most DB explorer tools (SQL workbench, Postico, etc).
  • Single source of truth documentation. It is easy to generate and push to documentation repositories (markdown, html, confluence). Either from Rails, CI, or any other tool in your workflow (see examples below).

Database Documentation in Postico

Postico OSX Postgres client showing comments as you explore DB structure

Code Samples

Below are some code samples to help you get started with a workflow around database documentation.

Migration

A migration adding comments to a previously existing table. You can add descriptions to call out deprecated fields, gotchas, planned refactorings, or add historical context that may be helpful to the next developer trying to understand what the field means.

class AddContactComments < ActiveRecord::Migration[5.1]
  def change
    msg = 'Contacts table holds individual details about our contacts, it is associated with leads and customers'
    change_table_comment(:contacts, msg)

    change_column_comment(:contacts, :first_name, 'the contacts first name')
    change_column_comment(:contacts, :last_name, 'the contacts last name')
    change_column_comment(:contacts, :house_latitude, 'the house_latitude the contact lives at')
    change_column_comment(:contacts, :house_longitude, 'the house_longitude the contact lives at')
    change_column_comment(:contacts, :house_location_accuracy, 'the accuracy range we captured the GPS with')
    change_column_comment(:contacts, :deleted_at, "the date the contact was 'hidden' from our DB")
    
    # call out gotchas
    # tasks have assignee_id while contacts still use agent_id in the DB, this is a recommended refactoring
    msg = 'the agent_id field is for who the contact is currently assigned to, various places in the code and API it is referenced by assignee_id'
    change_column_comment(:leads, :agent_id, msg)
  end
end

Ruby DB Docs Access

To generate documentation which could be pushed to a wiki, html, confluence, or elsewhere you can iterate through a tables columns and fetch the comments.

> ActiveRecord::Base.connection.table_comment('leads')
=> "Leads table holds individual details about leads, related associations, event timestamps, and joins to contact"	
> Contact.columns_hash['literacy'].comment
=> reading level: {"-1"=>"none", "0"=>"no_read", "1"=>"limited_read", "2"=>"read_fluent", "none"=>"none", "no_read"=>"no_read", "limited_read"=>"limited_read", "read_fluent"=>"read_fluent"}

# iterate through all the columns on a table and output them to your documentation file or API
Contact.columns.each do |c|
  puts c.comment 
end

SQL DB Docs Access

Obviously you don’t need Rails to get at this information, you can pull it out with raw SQL as well. Covered in this post: Querying table, view, column and function descriptions

# get the all the table comments in your DB
SELECT c.relname As tname, CASE WHEN c.relkind = 'v' THEN 'view' ELSE 'table' END As type, 
    pg_get_userbyid(c.relowner) AS towner, t.spcname AS tspace, 
    n.nspname AS sname,  d.description
   FROM pg_class As c
   LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
   LEFT JOIN pg_tablespace t ON t.oid = c.reltablespace
   LEFT JOIN pg_description As d ON (d.objoid = c.oid AND d.objsubid = 0)
   WHERE c.relkind IN('r', 'v') AND d.description > ''
   ORDER BY n.nspname, c.relname ;

# get all the comments for the contacts table
SELECT a.attname As column_name,  d.description
   FROM pg_class As c
    INNER JOIN pg_attribute As a ON c.oid = a.attrelid
   LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
   LEFT JOIN pg_tablespace t ON t.oid = c.reltablespace
   LEFT JOIN pg_description As d ON (d.objoid = c.oid AND d.objsubid = a.attnum)
   WHERE  c.relkind IN('r', 'v') AND  n.nspname = 'public' AND c.relname = 'contacts'
   ORDER BY n.nspname, c.relname, a.attname ;
comments
Dan Mayer Profile Pic
Welcome to Dan Mayer's development blog. I primary write about Ruby development, distributed teams, and dev/PM process. The archives go back to my first CS classes during college when I was first learning programming. I contribute to a few OSS projects and often work on my own projects, You can find my code on github.

Twitter @danmayer

Github @danmayer