Hi, I'm Rob17+ years of professional experience in web development, ecommerce, mobile, and product management with a love for SQL and data analysis.2024-01-15T08:05:27Zhttps://www.robkjohnson.comRob Johnsonhttps://www.robkjohnson.comA simple HTMX demo2024-01-14T00:00:00Zhttps://www.robkjohnson.com/posts/htmx-demo/<p>There's a new front-end tech that is making waves as <a href="https://risingstars.js.org/2023/en#section-framework">htmx finished 2nd in the 2023 JavaScript Rising Stars "Front-end Frameworks" category</a>, just behind React (htmx is a library, btw) and #10 overall!</p>
<p>After a quick review, I can see why. Below is a simple demo taken from their <a href="https://htmx.org/docs/">docs</a> to show you how native this is to standard HTML.</p>
<p>See the following markup with an explanation of what it does.</p>
<pre class="language-plain"><code class="language-plain"><span class="highlight-line"><button hx-get="/demo/htmx-demo.html"</span>
<span class="highlight-line"> hx-trigger="click"</span>
<span class="highlight-line"> hx-target="#htmx-demo"</span>
<span class="highlight-line"> hx-swap="outerHTML"</span>
<span class="highlight-line">></span>
<span class="highlight-line"> Try the demo </span>
<span class="highlight-line"></button></span></code></pre>
<blockquote>
<p>When a user clicks on this button, issue an HTTP GET request to ‘/demo/htmx-demo.html’ and use the content from the response to replace the element with the id "htmx-demo" in the DOM</p>
</blockquote>
<div id="htmx-demo">
<button class="button download" hx-get="/demo/htmx-demo.html" hx-trigger="click" hx-target="#htmx-demo" hx-swap="innerHTML">Try the demo</button>
</div>
<p>What's great is that all you need to use this framework is a 48kb file (current version 1.9.10 as of this post).</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/assets/js/htmx.1.9.10.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span></span></code></pre>
<p>You can download the latest htmx file at <a href="https://unpkg.com/browse/htmx.org@1.9.10/dist/">unpkg</a>.</p>
<p>The following HTTP request methods are available:</p>
<table>
<thead>
<tr>
<th>Attribute</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>hx-get</td>
<td>Issues a GET request to the given URL</td>
</tr>
<tr>
<td>hx-post</td>
<td>Issues a POST request to the given URL</td>
</tr>
<tr>
<td>hx-put</td>
<td>Issues a PUT request to the given URL</td>
</tr>
<tr>
<td>hx-patch</td>
<td>Issues a PATCH request to the given URL</td>
</tr>
<tr>
<td>hx-delete</td>
<td>Issues a DELETE request to the given URL</td>
</tr>
</tbody>
</table>
<p>I'm interested to use this more and see what it's like to live with. I really like the idea of leveraging modern capabilities of browsers instead of loading heavy frameworks on initial pageloads.</p>
How to get the template ID from a Shopify webpage for programatic use of the Section Rendering API2024-01-12T00:00:00Zhttps://www.robkjohnson.com/posts/shopify-get-template-id-section-id-section-rendering-api/<p>Shopify has a handy <a href="https://shopify.dev/docs/api/section-rendering">Section Rendering API</a> where you can pass the following parameters on a live page URL to grab the HTML for certain sections. This can be helpful for rendering the same section in another app or custom webpage elsewhere.</p>
<pre class="language-http"><code class="language-http"><span class="highlight-line">?sections=main-password-header,sections--1234__header</span>
<span class="highlight-line">?sections[]=main-password-header&sections[]=sections--1234__header</span></code></pre>
<p>An example of an absolute URL is:</p>
<pre class="language-http"><code class="language-http"><span class="highlight-line"><span class="token header"><span class="token header-name keyword">https</span><span class="token punctuation">:</span><span class="token header-value">//www.mystore.com/products/product-example/?sections=main-password-header`.</span></span></span></code></pre>
<p>The output from that endpoint will be a json object like this:</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"main-password-header"</span><span class="token operator">:</span> <span class="token string">"SECTION HTML (ESCAPED) RETURNED HERE"</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>For custom sections however, you have a URL that looks something like this.</p>
<pre class="language-http"><code class="language-http"><span class="highlight-line"><span class="token header"><span class="token header-name keyword">https</span><span class="token punctuation">:</span><span class="token header-value">//www.mystore.com/products/product-example/?sections=template--12345678912345__391178d3-08f5-48cb-865d-c407f193babe</span></span></span></code></pre>
<p>In the above URL the ID <code>12345678912345</code> is the template ID and <code>391178d3-08f5-48cb-865d-c407f193babe</code> is the section ID. The format of the rest of the parameter value is always the same.</p>
<p>This means in order to programatically request that section via the Section Rendering API, you need to know the template ID and the section ID. The problem is that there is no public endpoint that gives you the ID of the template ID, whereas the section ID's are all contained in the section .json file such as <code>product.product-example.json</code> at the bottom of the file in the <code>order</code> array. Example below.</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token property">"order"</span><span class="token operator">:</span> <span class="token punctuation">[</span></span>
<span class="highlight-line"> <span class="token string">"main"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"391178d3-08f5-48cb-865d-c407f193babe"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"8d85d3cc-b9ff-4341-be56-2122abc53ddb"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"7419dc99-9ec8-4010-8a81-d582506cc733"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"83547083-3690-41fc-8058-4cfe2d74e2a9"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"3914ae2c-4ce0-4926-9511-6491f2d9e5d4"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"f7b27d6e-3e10-4d48-b523-e2fa958ee6b1"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"197d4fc7-03ab-4150-b792-8612b3e3805b"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"2a632669-d028-4b6a-b261-0924191ef3a4"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"b02c4edf-751e-44ec-bd0e-776da702603c"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"d3acc40e-4c6d-47f5-afe9-2d1661380f9f"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"a4e4ce51-490f-45c9-bb40-0d3aecaba76c"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"86c2db3d-e9fd-4a43-b4d0-83f8f09b8ac3"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"d3a5d29b-a1af-4fcf-8221-2545d2930f9d"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"e7ca7e37-b418-47a6-8bc0-62d9cd68a4e1"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"766ae22c-cac2-477d-a900-cbf424b6bca5"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"e3dc966d-fff0-446f-a36d-9d2e79c8b769"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"398c67c9-9a4b-4d8b-bb11-f6dd8f5b83df"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"related-products"</span></span>
<span class="highlight-line"><span class="token punctuation">]</span></span></code></pre>
<h3>Solution 1</h3>
<p>You can run this javascript that will extract the template ID from a page by looking at the first <code><section></code> tag and parsing out the template ID integer from the 'id' for that section.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">extractTemplateId</span><span class="token punctuation">(</span><span class="token parameter">id</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">const</span> prefix <span class="token operator">=</span> <span class="token string">'shopify-section-template--'</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token keyword">const</span> startIndex <span class="token operator">=</span> id<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span>prefix<span class="token punctuation">)</span> <span class="token operator">+</span> prefix<span class="token punctuation">.</span>length<span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token keyword">const</span> endIndex <span class="token operator">=</span> id<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">'__'</span><span class="token punctuation">,</span> startIndex<span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token keyword">const</span> numberPart <span class="token operator">=</span> id<span class="token punctuation">.</span><span class="token function">substring</span><span class="token punctuation">(</span>startIndex<span class="token punctuation">,</span> endIndex<span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">parseInt</span><span class="token punctuation">(</span>numberPart<span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// get the first shopify section ID for extraction</span></span>
<span class="highlight-line"><span class="token keyword">const</span> first_section_id <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">querySelector</span><span class="token punctuation">(</span><span class="token string">'main section.shopify-section'</span><span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token keyword">const</span> fullId <span class="token operator">=</span> first_section_id<span class="token punctuation">.</span>id</span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// Run the extractTemplateId function</span></span>
<span class="highlight-line"><span class="token keyword">const</span> shopifyTemplateId <span class="token operator">=</span> <span class="token function">extractTemplateId</span><span class="token punctuation">(</span>fullId<span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>shopifyTemplateId<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Output example: 12345678912345</span></span></code></pre>
<h3>Solution 2 (recommended)</h3>
<p>If you have control of the Shopify theme, you can add the following code to <code>main-product.liquid</code>.</p>
<pre class="language-plaintext"><code class="language-plaintext"><span class="highlight-line">{% liquid </span>
<span class="highlight-line"> assign section_id = 'template--12345678912345__391178d3-08f5-48cb-865d-c407f193babe'</span>
<span class="highlight-line"> assign section_id_parts = section_id | split: 'template--'</span>
<span class="highlight-line"> assign section_with_id = section_id_parts[1]</span>
<span class="highlight-line"> assign section_id_parts = section_with_id | split: '__'</span>
<span class="highlight-line"> assign template_id = section_id_parts[0]</span>
<span class="highlight-line">%}</span></code></pre>
<p>And then add the <code>data-template_id</code> attribute to the <code><section></code> block seen here on line <strong>5</strong>:</p>
<pre class="language-plaintext"><code class="language-plaintext"><span class="highlight-line"><section</span>
<span class="highlight-line"> id="MainProduct-{{ section.id }}"</span>
<span class="highlight-line"> class="page-width section-{{ section.id }}-padding"</span>
<span class="highlight-line"> data-section="{{ section.id }}"</span>
<span class="highlight-line"> data-template_id="{{ template_id }}"</span>
<span class="highlight-line">></span></code></pre>
<p>This will then allow you to grab the template_id for each page with the following simple JS code:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">document<span class="token punctuation">.</span><span class="token function">querySelector</span><span class="token punctuation">(</span><span class="token string">'section[data-section]'</span><span class="token punctuation">)</span><span class="token punctuation">.</span>dataset<span class="token punctuation">.</span>template_id</span>
<span class="highlight-line"><span class="token comment">// output example: 12345678912345</span></span></code></pre>
<h3>Warning</h3>
<p>Both solution 1 and 2 depend on the default string that Shopify generate for a section ID. This could change at any given time although not likely. If you're wanting to do something programmatic, make sure to add a fallback or log/alert if this ever fails.</p>
The infrastructure that powers the worlds biggest and fastest fulfillment network2023-09-23T00:00:00Zhttps://www.robkjohnson.com/posts/amazons-warehouse-fulfillment-network-scale/<p>Have you ever wondered how large of an operation is needed to fulfill all of Amazon's orders at a rate that is 50% faster than all its retail competitors?</p>
<p>Meet <a href="https://supplychain.amazon.com/mcf">Amazon's Multi-Channel Fulfillment</a>, a growing and strategic part of Amazon's business.</p>
<p>Like most of Amazon's services, once they have figured it out for themselves, they productize it and sell it to the open market, similar to Amazon Web Services (AWS).</p>
<blockquote>
<p>Amazon MCF is a third-party logistics (3PL) solution that enables you to leverage Amazon’s fulfillment network and team of experts to pick, pack, ship, and deliver your customer orders from off-Amazon sales channels.</p>
</blockquote>
<hr />
<p><img src="https://www.robkjohnson.com/assets/img/content/amazon-mcf.png#xx-small" alt="Amazon's Multi-channel fulfillment" /></p>
<section class="content">
<ul class="cards stats">
<li>
<data>2000</data>
<label>Facilities inc. 200 fulfillment centers</label>
</li>
<li>
<data>~160<sup>M</sup></data>
<label>sq ft warehouse space</label>
</li>
<li>
<data>1.27<sup>M</sup></data>
<label>staff + partners</label>
</li>
<li>
<data>120<sup>K</sup></data>
<label>Trucks, vans, and planes</label>
</li>
<li>
<data>10<sup>B</sup></data>
<label>Annual packages</label>
</li>
<li>
<data>1.9</data>
<label>days avg fulfillment</label>
</li>
<li>
<data>97<sup>%</sup></data>
<label>on-time delivery rate</label>
</li>
<li>
<data>99.98<sup>%</sup></data>
<label>undamaged delivery rate</label>
</li>
</ul>
</section>
<hr />
<p>Part of the above exceptional performance is the investment Amazon has made in robotics that started more than <a href="https://www.amazon.science/latest-news/amazon-robotics-autonomous-drive-units-hercules-pegasus-xanthus-xbot">10 years ago</a>.
A typical Amazon fulfillment center contains fleets of robotic drives, autonomous mobile robots that transport goods.</p>
<section class="content">
<ul class="cards stats">
<li>
<data>750<sup>K</sup></data>
<label>robotic drive units</label>
</li>
<li>
<data>10<sup>YEARS</sup></data>
<label>developing robotics</label>
</li>
</ul>
</section>
<h3>Meet the robots</h3>
<ol>
<li><strong>Hercules</strong>: A fourth-generation drive designed to navigate structured fields, floors that contain a grid of encoded markers. By reading the markers with its downward facing camera, it can find its position and the location of any pod.</li>
<li><strong>Pegasus</strong>, whose name evokes the winged horse of Greek mythology, sorts parcels by zip code or delivery route.</li>
<li><strong>Xanthus</strong>, named for the immortal horse that drew Achilles’ chariot, also sorts but can do other tasks as well.</li>
</ol>
<p><img src="https://www.robkjohnson.com/assets/img/content/amazon-hercules-demo.gif#small" alt="Amazon's Multi-channel fulfillment" /></p>
<p class="caption is-centered">Hercules can lift up to 1,250 pounds and travel across the 1 million square feet.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/amazon-hercules.jpg#small" alt="Amazon's Hercules warehouse robot" /></p>
<p class="caption is-centered">Amazon is currently using more than 750,000 robotic drive units around the world. Pictured above is Hercules.</p>
<!-- ![Amazon's Multi-channel fulfillment](/assets/img/content/amazon-robotic-drives.webp#small)
<p class="caption is-centered">Hercules robots — which can lift 1,250 pounds</p> -->
<ul>
<li>Amazon is currently using more than 750,000 robotic drive units around the world.</li>
<li>Hercules is used to pickup product shelves and deliver them to employees.</li>
<li>Robotic drive units leverage AI to create routes using 2D lines and barcodes.</li>
<li>Computer vision is used to inspect each inventory pod after delivery to ensure no products are at risk of falling out or placed in the wrong bin.</li>
</ul>
<section class="content divide">
<h3>How MCF's inventory works</h3>
<div class="steps" id="steps">
<div class="step">
<div class="timeline_number">1</div>
<div class="info">
<p class="title">Send your inventory to Amazon</p>
<p class="text">Ship your stock to one of Amazon's locations. Amazon will then distribute your stock across their network of fulfillment centers. This is part of their not-so-secret sauce for speedy delivery.</p>
</div>
</div>
<div class="step">
<div class="timeline_number completed">2</div>
<div class="info">
<p class="title">Send fulfillment requests via Amazon MCF's API</p>
<p class="text">Build or buy a connection service that submits orders and checks on the fulfillment and delivery status of your fulfillment requests. Shopify have an app, or you can rely on other Order Management Solutions. One of my favorites for Amazon MCF is [Pipe17](https://pipe17.com</p>
</div>
</div>
<div class="step">
<div class="timeline_number completed">3</div>
<div class="info">
<p class="title">Amazon receives, picks and packs your order</p>
<p class="text">As a merchant, you now have the ability to scale and perform like Amazon. Sit back and let them take care of the heavy lifting.</p>
</div>
</div>
<div class="step">
<div class="timeline_number completed">4</div>
<div class="info">
<p class="title">Amazon then delivers the order</p>
<p class="text">Benefit from Amazon's speedy delivery service that is 50% faster than other retailers. Amazon just announced that they have launched a courier service, so technically you could get this part of the service without leveraging their warehouses for pick/pack.</p>
</div>
</div>
</div>
</section>
<h3>A business example of an effective rate</h3>
<section class="content">
<ul class="cards stats">
<li>
<data>6-12<sup>OZ</sup></data>
<label>per item weight</label>
</li>
<li>
<data>1/4<sup>CSF</sup></data>
<label>per item storage size</label>
</li>
<li>
<data>10<sup>K</sup></data>
<label>Monthly orders</label>
</li>
<li>
<data>60<sup>K</sup></data>
<label>AVG units stored</label>
</li>
<li class="is-danger">
<data><sup>$</sup>18,788</data>
<label>Monthly Storage + Fulfillment + Delivery cost</label>
</li>
</ul>
</section>
<p>Let's assume you sell a product for $30 and that your average units per transaction is 2. You sell approx 10,000 units per month and the space that each product takes up is 1/4 of a cubic foot, each unit being between 6 and 12oz.</p>
<p>Your costs would be</p>
<ul>
<li>Fulfillment cost per order: <strong>$5.49</strong></li>
<li>Total fulfillment cost per month: <strong>$54,900</strong></li>
<li>The blended storage cost for ~60k units each month (2 rates, higher for Q4): <strong>$18,787.50</strong></li>
</ul>
<p>Your revenue would be <strong>$600,000</strong></p>
<p>Your margin cost would be ($67,950 / $600,000) * 100 = <strong>12.28%</strong></p>
<p>You can see that the cost per month for Q4 is <strong>176%</strong> higher than Jan - Sept.</p>
<ul>
<li>Jan - Sept monthly cost: $13,050</li>
<li>Oct - Dec monthly cost: $36,000</li>
</ul>
<p>Annual Revenue: <strong>$7,200,000</strong></p>
<p>Annual Total cost = $658,800 (fulfillment) + $225,450 (storage) = <strong>$884,250</strong></p>
<section class="content">
<ul class="cards stats">
<li>
<data>30<sup>$</sup></data>
<label>per item MSRP</label>
</li>
<li class="is-success">
<data><sup>$</sup>7.2<sup>M</sup></data>
<label>Annual Revenue</label>
</li>
<li class="is-danger">
<data>12.28<sup>%</sup></data>
<label>Margin cost</label>
</li>
<li class="is-info">
<data><sup>$</sup>6.316<sup>M</sup></data>
<label>Annual Net revenue</label>
</li>
</ul>
</section>
<hr />
<h3>Amazon drones: The future of delivery?</h3>
<p>Below is an Amazon drone I snapped a picture of from this years <a href="https://amazonaccelerate.com/pre-event/">Amazon Accelerate</a>. Nothing official has been announced yet, but you can be certain that years of development have already been happening and that drone delivery will be a very likely method of fulfillment someday soon.</p>
<p>The obvious connection in autonomous delivery is the lower cost of delivery. With new autonomous tech like this, Amazon can make more margin and continue to be leaps and bounds ahead of the competition.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/amazon-drone-delivery.jpg#small" alt="Amazon's Drone Delivery" /></p>
<p class="caption is-centered">Amazon's drone delivery aims to get your order to you within 1 hour.</p>
<hr />
<h3>Considerations for 3PL fulfillment</h3>
<p>If you're experiencing unexpected or prolific growth for your ecommerce DTC channel, the importance of scalable operations for fulfillment cannot be overstated. Rapid expansion can put immense pressure on supply chains, inventory management, and customer service. Therefore, it's crucial to have an agile, scalable ecommerce fulfillment strategy in place from day one. This ensures that as your customer base grows, your operations can smoothly scale to meet demand without sacrificing efficiency or customer satisfaction. A well-planned, scalable fulfillment system can make the difference between capitalizing on growth opportunities and crumbling under operational inefficiencies.</p>
<p>There is a balance to strike between internal resources, organizational core competencies, the scale of your operation, the speed of fulfillment and the average cost per fulfillment.</p>
<hr />
<p>If you're interested, you can see 2023's rate card info below or go <a href="https://supplychain.amazon.com/blog/price-and-calculate-shipping">download Amazon's rate card here</a>.</p>
<h3>2023 Standard Rates</h3>
<table>
<thead>
<tr>
<th>Size Tier</th>
<th>Weight</th>
<th>1 Unit Order</th>
<th>2 Unit Order</th>
<th>3 Unit Order</th>
<th>4+ Unit Order</th>
</tr>
</thead>
<tbody>
<tr>
<td>Small standard</td>
<td>6 oz or less</td>
<td>$7.15</td>
<td>$4.92</td>
<td>$4.19</td>
<td>$3.72</td>
</tr>
<tr>
<td>Small standard</td>
<td>6+ to 12 oz</td>
<td>$7.80</td>
<td>$5.49</td>
<td>$4.72</td>
<td>$4.23</td>
</tr>
<tr>
<td>Small standard</td>
<td>12+ to 16 oz</td>
<td>$8.25</td>
<td>$5.70</td>
<td>$4.88</td>
<td>$4.38</td>
</tr>
<tr>
<td>Large standard</td>
<td>6 oz or less</td>
<td>$7.35</td>
<td>$5.05</td>
<td>$4.49</td>
<td>$4.08</td>
</tr>
<tr>
<td>Large standard</td>
<td>6+ to 12 oz</td>
<td>$8.20</td>
<td>$5.80</td>
<td>$4.90</td>
<td>$4.50</td>
</tr>
<tr>
<td>Large standard</td>
<td>12+ to 16 oz</td>
<td>$8.50</td>
<td>$6.05</td>
<td>$5.15</td>
<td>$4.70</td>
</tr>
<tr>
<td>Large standard</td>
<td>1+ to 2 lbs</td>
<td>$9.50</td>
<td>$6.65</td>
<td>$5.88</td>
<td>$4.75</td>
</tr>
<tr>
<td>Large standard</td>
<td>2+ to 20 lbs</td>
<td>$9.50 + $0.62/lb above first 2 lb</td>
<td>$6.65 + $0.55/lb above first 2 lb</td>
<td>$5.88 + $0.55/lb above first 2 lb</td>
<td>$4.75 + $0.55/lb above first 2 lb</td>
</tr>
<tr>
<td>Small oversize</td>
<td>2+ to 30 lbs</td>
<td>$16.00 + $0.62/lb above first 2 lb</td>
<td>$11.79 + $0.55/lb above first 2 lb</td>
<td>$10.18 + $0.55/lb above first 2 lb</td>
<td>$8.62 + $0.55/lb above first 2 lb</td>
</tr>
<tr>
<td>Small oversize</td>
<td>Over 30 lb</td>
<td>$32.88 + $0.62/lb above first 30 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>Medium oversize</td>
<td>N/A</td>
<td>$25.25 + $0.62/lb above first 2 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>Large oversize</td>
<td>N/A</td>
<td>$118.80 + $1.16/lb above first 90 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>Special oversize</td>
<td>N/A</td>
<td>$189.19 + $1.21/lb above first 90 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
</tbody>
</table>
<h3>2023 Peak Rates</h3>
<table>
<thead>
<tr>
<th>Size Tier</th>
<th>Weight</th>
<th>1 Unit Order</th>
<th>2 Unit Order</th>
<th>3 Unit Order</th>
<th>4+ Unit Order</th>
</tr>
</thead>
<tbody>
<tr>
<td>Small standard</td>
<td>6 oz or less</td>
<td>$7.35</td>
<td>$5.12</td>
<td>$4.39</td>
<td>$3.92</td>
</tr>
<tr>
<td>Small standard</td>
<td>6+ to 12 oz</td>
<td>$8.00</td>
<td>$5.69</td>
<td>$4.92</td>
<td>$4.43</td>
</tr>
<tr>
<td>Small standard</td>
<td>12+ to 16 oz</td>
<td>$8.45</td>
<td>$5.90</td>
<td>$5.08</td>
<td>$4.58</td>
</tr>
<tr>
<td>Large standard</td>
<td>6 oz or less</td>
<td>$7.65</td>
<td>$5.35</td>
<td>$4.79</td>
<td>$4.38</td>
</tr>
<tr>
<td>Large standard</td>
<td>6+ to 12 oz</td>
<td>$8.50</td>
<td>$6.10</td>
<td>$5.20</td>
<td>$4.80</td>
</tr>
<tr>
<td>Large standard</td>
<td>12+ to 16 oz</td>
<td>$8.80</td>
<td>$6.35</td>
<td>$5.45</td>
<td>$5.00</td>
</tr>
<tr>
<td>Large standard</td>
<td>1+ to 2 lbs</td>
<td>$9.80</td>
<td>$6.95</td>
<td>$6.18</td>
<td>$5.05</td>
</tr>
<tr>
<td>Large standard</td>
<td>2+ to 20 lbs</td>
<td>$10.00 + $0.62/lb above first 2 lb</td>
<td>$7.15 + $0.55/lb above first 2 lb</td>
<td>$6.38 + $0.55/lb above first 2 lb</td>
<td>$5.25 + $0.55/lb above first 2 lb</td>
</tr>
<tr>
<td>Small oversize</td>
<td>2+ to 30 lbs</td>
<td>$17.00 + $0.62/lb above first 2 lb</td>
<td>$12.79 + $0.55/lb above first 2 lb</td>
<td>$11.18 + $0.55/lb above first 2 lb</td>
<td>$9.62 + $0.55/lb above first 2 lb</td>
</tr>
<tr>
<td>Small oversize</td>
<td>Over 30 lb</td>
<td>$33.88 + $0.62/lb above first 30 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>Medium oversize</td>
<td>N/A</td>
<td>$27.75 + $0.62/lb above first 2 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>Large oversize</td>
<td>N/A</td>
<td>$121.30 + $1.16/lb above first 90 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
<tr>
<td>Special oversize</td>
<td>N/A</td>
<td>$191.69 + $1.21/lb above first 90 lb</td>
<td>N/A</td>
<td>N/A</td>
<td>N/A</td>
</tr>
</tbody>
</table>
<h3>Storage Rates</h3>
<h4>Non-dangerous goods products</h4>
<table>
<thead>
<tr>
<th>Month</th>
<th>Standard-size</th>
<th>Oversize</th>
</tr>
</thead>
<tbody>
<tr>
<td>January-September</td>
<td>$0.87 per cubic foot</td>
<td>$0.56 per cubic foot</td>
</tr>
<tr>
<td>October-December</td>
<td>$2.40 per cubic foot</td>
<td>$1.20 per cubic foot</td>
</tr>
</tbody>
</table>
Shopify Permalinks: Increase conversions and subscription attachment2023-09-18T00:00:00Zhttps://www.robkjohnson.com/posts/shopify-cart-permalinks-guide-selling-plans/<p>Are you looking to make the most out of your Shopify store's marketing and enhance your sales? Shopify has introduced a powerful new feature that allows you to create cart permalinks. This means I can now direct customers to my Shopify checkouts or online store cart with pre-loaded items. What's more, I can also append query parameters to these permalinks to include additional information and attribute orders to my sales channel. In this guide, I'll walk you through the process of creating these cart permalinks and even show you how to add a selling plan and a discount code to an item.</p>
<h3>Creating Cart Permalinks</h3>
<pre class="language-html"><code class="language-html"><span class="highlight-line">https://yourshopifystore.com/cart/1234:1,5678:1</span></code></pre>
<p>In this URL, you can see that I'm adding items to the cart. Each item is represented by a unique identifier followed by a colon and the quantity. In this case, I'm adding two items to the cart. It's as straightforward as that! Customers clicking on this link will find their cart pre-populated with the selected items.</p>
<h3>Adding Additional Information</h3>
<p>But what if you want to do more with your cart permalinks? You can append query parameters to the URL to include additional information. Here's an example:</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line">https://yourshopifystore.com/cart/{variant_id_1234}:1,{variant_id_5678}:1?selling_plan=555&discount={DISCOUNTCODEHERE}</span></code></pre>
<p>In this URL, I've added two query parameters: <code>selling_plan</code> and <code>discount</code>. This is where the magic happens. By specifying a selling plan, you can attach a subscription to the line item and adding a discount code can help with increasing your checkout conversion rate.</p>
<h3>What About Selling Plans?</h3>
<p>One thing to note is that Shopify's documentation might not cover how to add a selling plan to an item in detail. However, with the method I've shown you, you can easily incorporate selling plans into your cart permalinks. Just make sure to include the <code>selling_plan</code> query parameter followed by the plan's identifier.</p>
<h3>Conclusion</h3>
<p>Shopify's new cart permalink feature is great for creating simple URL's that provide a seamless shopping experience for customers. By creating custom cart permalinks with added information like selling plans and discount codes, you can take your e-commerce strategy to the next level. So, go ahead, experiment with these powerful links, and see how it help improve your Shopify store metrics!</p>
<p>For more detailed information, you can refer to the <a href="https://shopify.dev/docs/apps/checkout/cart-permalinks/cart-permalinks">official Shopify documentation</a>.</p>
<hr />
<div class="callout">
Shopify had an old way of creating permalinks. <a href="https://www.robkjohnson.com/posts/shopify-add-multple-items-to-cart-with-one-link/" title="See the updated article for Shopify permalinks">Check it out »</a>
</div>
Attaching sellings plans and other attributes to Shopify forms via the Liquid API2023-09-17T00:00:00Zhttps://www.robkjohnson.com/posts/shopify-forms-adding-selling-plans-attributes/<h3>Liquid API</h3>
<p>If you're using a native Shopify theme directly on Shopify, you can use the Liquid API. You can insert input fields directly into the product form, such as <code>selling_plan</code> which will be included in the form submission.</p>
<pre class="language-liquid"><code class="language-liquid"><span class="highlight-line"></span>
<span class="highlight-line"><span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">form</span> <span class="token string">'product'</span><span class="token punctuation">,</span> <span class="token object">product</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>hidden<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>id<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> current_variant<span class="token punctuation">.</span>id <span class="token delimiter punctuation">}}</span></span><span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>hidden<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>selling_plan<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> current_selling_plan_allocation<span class="token punctuation">.</span><span class="token object">selling_plan</span><span class="token punctuation">.</span>id <span class="token operator">|</span> <span class="token function filter">default</span><span class="token operator">:</span> <span class="token string">''</span> <span class="token delimiter punctuation">}}</span></span><span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>fieldset</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>legend</span><span class="token punctuation">></span></span>Product options<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>legend</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">for</span> option <span class="token keyword">in</span> <span class="token object">product</span><span class="token punctuation">.</span>options_with_values <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>label</span><span class="token punctuation">></span></span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> option<span class="token punctuation">.</span>name <span class="token delimiter punctuation">}}</span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>label</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>select</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">for</span> value <span class="token keyword">in</span> option<span class="token punctuation">.</span>values <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>option</span><span class="token punctuation">></span></span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> value <span class="token delimiter punctuation">}}</span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>option</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endfor</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>select</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endfor</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>fieldset</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment"><!-- Pass the product object as JSON so it can be used to update selling plan information using JavaScript --></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>fieldset</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>selling-plan-fieldset<span class="token punctuation">"</span></span> <span class="token attr-name">data-product</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> <span class="token object">product</span> <span class="token operator">|</span> <span class="token function filter">json</span> <span class="token delimiter punctuation">}}</span></span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>legend</span><span class="token punctuation">></span></span>Purchase options<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>legend</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">unless</span> <span class="token object">product</span><span class="token punctuation">.</span>requires_selling_plan <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>radio<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>purchase_option<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> One-time purchase</span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endunless</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">for</span> <span class="token object">group</span> <span class="token keyword">in</span> <span class="token object">product</span><span class="token punctuation">.</span>selling_plan_groups <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>radio<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>purchase_option<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> <span class="token object">group</span><span class="token punctuation">.</span>name<span class="token delimiter punctuation">}}</span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">for</span> option <span class="token keyword">in</span> <span class="token object">group</span><span class="token punctuation">.</span>options <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>label</span><span class="token punctuation">></span></span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> option<span class="token punctuation">.</span>name <span class="token delimiter punctuation">}}</span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>label</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>select</span> <span class="token attr-name">data-position</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> option<span class="token punctuation">.</span>position <span class="token delimiter punctuation">}}</span></span><span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">for</span> value <span class="token keyword">in</span> option<span class="token punctuation">.</span>values <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>option</span><span class="token punctuation">></span></span><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span> value <span class="token delimiter punctuation">}}</span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>option</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endfor</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>select</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endfor</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endfor</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>fieldset</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token liquid language-liquid"><span class="token delimiter punctuation">{%</span> <span class="token keyword">endform</span> <span class="token delimiter punctuation">%}</span></span></span>
<span class="highlight-line"></span></code></pre>
<p>You can also add in customer line item properties with the following HTML which will pass through when the form is submitted.</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>text<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>properties[Engraving]<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span></code></pre>
<p>You can do the same thing with hidden properties which won't be visible to the customer by prefixing the <code>value=</code> with an underscore <code>_</code> and using <code>type="hidden"</code> on the input field you can hide the line item property on the form also.</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>hidden<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>properties[_cohort]<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cohort_1<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span></code></pre>
Speed up your storefront by loading scripts on demand2023-03-10T00:00:00Zhttps://www.robkjohnson.com/posts/speed-up-your-storefront-lower-network-cost/<p>Some services require you to load their widget on every page, even if it's unlikely that a user will use it. An example of this is a customer support widget. Although a customer may need to use it at some point, most page views won't require it.</p>
<p>This can be problematic if the widget is a floating widget that may interfere with important elements on your page, such as the add-to-cart button. Additionally, loading the widget on every page can increase the total load time and size of each page. This can negatively impact both the customer's experience, especially if they're on a mobile internet connection, and your website's SEO due to slowing down each page.</p>
<p>In this example, we're loading a customer support widget from <a href="https://explore.zoom.us/en/products/contactcenter/features/virtual-agent/?solvvy=1">Solvvy</a>. To ensure the widget only loads when a customer wants to use it, I've written the following JavaScript function which checks whether the widget has already loaded. If it hasn't, the function adds the script tag to the page and executes the JavaScript.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">launchSolvvy</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span> window<span class="token punctuation">.</span>Solvvy <span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> Solvvy<span class="token punctuation">.</span><span class="token function">open</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment">// Create a script element</span></span>
<span class="highlight-line"> <span class="token keyword">const</span> script <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">'script'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> script<span class="token punctuation">.</span>src <span class="token operator">=</span> <span class="token string">'https://cdn.solvvy.com/deflect/customization/{name}/solvvy.js'</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment">// Append the script element to the document head</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span>head<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>script<span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">"solvvy_ready"</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Solvvy widget loaded"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token comment">// Open chat window</span></span>
<span class="highlight-line"> Solvvy<span class="token punctuation">.</span><span class="token function">open</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Then I created a simple link for "Chat" with the following HTML.</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token special-attr"><span class="token attr-name">onclick</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token value javascript language-javascript"><span class="token function">launchSolvvy</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span><span class="token punctuation">"</span></span></span> <span class="token attr-name">title</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Open Chat<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>Chat<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>a</span><span class="token punctuation">></span></span></span></code></pre>
<p>Instead of loading another webpage, you can use an <code><a></code> tag to initiate the <code>launchSolvvy()</code> function that we just added.</p>
<h3>Once the script is loaded, you can use the services API's as you need</h3>
<p>This particular widget from <a href="https://explore.zoom.us/en/products/contactcenter/features/virtual-agent/?solvvy=1">Solvvy</a> has an <a href="https://developers.solvvy.com/web-widget-api/getting-started#usage">API</a> tha can perform certain functions. This includes methods to <a href="https://developers.solvvy.com/web-widget-api/api-methods/open">open and close the widget</a>. Once we know the widget has loaded by listening for the <code>solvvy_ready</code> event, when can then open the widget by simply calling <code>Solvvy.open()</code>.</p>
<h3>User Experience</h3>
<p>This approach leads to faster page load times for customers, and the widget will load immediately when they click the link. To improve the user experience further, you can add a loading icon to the page once the link is pressed, in case the request for the javascript file to load and execute takes a few seconds.</p>
Create a custom Product Conversion Funnel Report with ShopifyQL Notebooks2023-03-09T00:00:00Zhttps://www.robkjohnson.com/posts/create-custom-product-conversion-funnel-report-shopifyql-notebooks/<p>Shopify released <a href="https://www.shopify.com/editions/summer2022#search-discovery-app">ShopifyQL Notebooks in Summer 2022</a>. As a beta group member, I had the opportunity to provide feedback directly to the Product team.</p>
<p>I was immediately onboard with this product, as it can be a powerful tool for querying data directly within your Shopify instance without requiring a Data Warehouse connection/service such as <a href="https://www.snowflake.com/en/">Snowflake</a> or <a href="https://powerbi.microsoft.com/en-us/">PowerBI</a>, which can be very painful if you are used to having direct access to your storefront's database for querying.</p>
<p>ShopifyQL Notebooks definitely has its limitations. However, it can be very helpful in quickly grabbing accurate data if it's readily available in their data model set. Also, it's not true SQL syntax and does not have the same verbose options as traditional SQL options such as Postgres or MySQL.</p>
<h3>ShopifyQL Notebook SQL for funnel conversion by product</h3>
<p>The query below provides insights into the performance of products, including their add-to-cart rates, checkout rates, conversion rates, net sales, and average quantity purchased. This information can help you quickly identify the best-performing products for your store, enabling you to optimize your merchandising and product placement strategies and increase your conversion rates and revenue.</p>
<p>As you can see, the below statement is not true SQL syntax. ShopifyQL doesn't have the same verbose options as traditional SQL options such as Postgres or MySQL. Here is the <a href="https://help.shopify.com/en/manual/reports-and-analytics/shopifyql-notebooks/using-shopifyql-for-notebooks">ShopifyQL syntax reference</a>.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">FROM</span> products</span>
<span class="highlight-line"><span class="token keyword">SHOW</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> views<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> add_to_carts<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_sessions<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> add_rate<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> checkouts<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_sessions<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> checkout_rate<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_purchase_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> purchases<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_purchase_sessions<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> conversion_rate<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>net_sales<span class="token punctuation">)</span> <span class="token keyword">AS</span> net_sales<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>quantity_purchased<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>purchase_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> avg_qty</span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span> product_title</span>
<span class="highlight-line">SINCE last_month UNTIL yesterday</span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span> net_sales <span class="token keyword">DESC</span></span>
<span class="highlight-line"><span class="token keyword">LIMIT</span> <span class="token number">50</span></span></code></pre>
<h3>ShopifyQL Notebook SQL for funnel conversion by Product Type</h3>
<p>This query performs the same calculations as before, but does a <code>GROUP BY product_type</code> instead of <code>product_title</code>.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">FROM</span> products</span>
<span class="highlight-line"><span class="token keyword">SHOW</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> views<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> add_to_carts<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_sessions<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> add_rate<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> checkouts<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_sessions<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> checkout_rate<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_purchase_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> purchases<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>view_cart_checkout_purchase_sessions<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>view_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> conversion_rate<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>net_sales<span class="token punctuation">)</span> <span class="token keyword">AS</span> net_sales<span class="token punctuation">,</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>quantity_purchased<span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token function">sum</span><span class="token punctuation">(</span>purchase_sessions<span class="token punctuation">)</span> <span class="token keyword">AS</span> avg_qty</span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span> product_type</span>
<span class="highlight-line">SINCE last_month UNTIL yesterday</span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span> net_sales <span class="token keyword">DESC</span></span>
<span class="highlight-line"><span class="token keyword">LIMIT</span> <span class="token number">20</span></span></code></pre>
<h3>Conclusion</h3>
<p>ShopifyQL Notebooks is a developing product with a lot of potential. I hope Shopify continues to build this tool out. While it's very limited and only has one simple graph visualization option, it's free, quick, and easy to use. It has the potential to be so much more, but it's not currently Enterprise-grade and will likely never be. I still use Snowflake for most deep-dive analysis. However, if you have access, it's definitely worth a try and can be a great solution for providing reports that internal teams may benefit from without the need of an additional service license to a solution such as Snowflake.</p>
Accessing the Browser Performance API to Prevent Showing Back/Forward Page Cache2023-01-06T00:00:00Zhttps://www.robkjohnson.com/posts/handling-browser-back-button-cache/<p>All modern browsers are configured to ensure you have the fastest loading times possible.</p>
<p>One of the methods of speeding up load times is to save a local cache of the page contents of the "Back" and "Next" pages that you've recently visited.</p>
<h3>The Problem</h3>
<p>If someone logs out of your application, from an account page, then the user can click the browser "Back" button after logging out and view a locally cached version of the account page.</p>
<h3>Server-side Solution (Ideal)</h3>
<p>Ideally you want the server to communicate to the browser on the initial 200 request that the browser should not use cache to render the page. This can be done with <code>Control-Cache</code> directives in document header.</p>
<p>If you have access to the headers of the 200 document, then it's much easier to use <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control">the following headers</a> to ensure local cache isn't used. See also a great <a href="https://developers.cloudflare.com/cache/about/cache-control">explanation from CloudFlare about Origin Cache Control</a>.</p>
<h3>A Javascript Solution</h3>
<p>If you don't have access to change the server-side cache-control headers, you can leverage information from the browser that tracks if the user accessed the page via the "Back" or "Forward" button.</p>
<p>How can you detect if a user is accessing a page from using the "Back" or "Forward" buttons?</p>
<p><code>PerformanceNavigationTiming</code> is part of a modern browsers <a href="https://developer.mozilla.org/en-US/docs/Web/API/PerformanceNavigationTiming/type">Performance API</a> which captures user navigational data.</p>
<p>Inside of the performance interface, there is a <code>navigation</code> object that stores various types of entries which include:</p>
<ol>
<li>navigate</li>
<li>reload</li>
<li>back_forward</li>
<li>prerender</li>
</ol>
<p>Add an EventListener to the pageshow, then check for the type 'back_forward' and reload the page if TRUE. The reload will ensure that the cache is refreshed.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">window<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span> <span class="token string">"pageshow"</span><span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> <span class="token parameter">event</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>window<span class="token punctuation">.</span>performance<span class="token punctuation">.</span><span class="token function">getEntriesByType</span><span class="token punctuation">(</span><span class="token string">"navigation"</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>type <span class="token operator">==</span> <span class="token string">'back_forward'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> window<span class="token punctuation">.</span>location<span class="token punctuation">.</span><span class="token function">reload</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>You can also use HTTP META tags, but in some cases this may not be sufficient.</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">http-equiv</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Cache-Control<span class="token punctuation">"</span></span> <span class="token attr-name">content</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>no-cache, no-store, must-revalidate, max-age=0, private, post-check=0, pre-check=0<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">http-equiv</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Pragma<span class="token punctuation">"</span></span> <span class="token attr-name">content</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>no-cache<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">http-equiv</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Expires<span class="token punctuation">"</span></span> <span class="token attr-name">content</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>0<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span></code></pre>
Add multiple products, subscriptions and qty to a Shopify cart, via Javascript Cart API2022-09-30T00:00:00Zhttps://www.robkjohnson.com/posts/shopify-add-to-cart-mulitple-products-selling-plans-via-javascript-cart-api/<p>You can use the following javascript which uses <a href="https://shopify.dev/api/ajax/reference/cart">Shopify's Cart API</a> to add multiple products to your cart with one click of a button.</p>
<h2>Javascript function to add to global.js</h2>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// add to cart function that keeps customer on the same page</span></span>
<span class="highlight-line">Shopify<span class="token punctuation">.</span><span class="token function-variable function">addToCart</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token parameter">items</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token function">fetch</span><span class="token punctuation">(</span>window<span class="token punctuation">.</span>Shopify<span class="token punctuation">.</span>routes<span class="token punctuation">.</span>root <span class="token operator">+</span> <span class="token string">'cart/add.js'</span><span class="token punctuation">,</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token literal-property property">method</span><span class="token operator">:</span> <span class="token string">'POST'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">headers</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'Content-Type'</span><span class="token operator">:</span> <span class="token string">'application/json'</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">body</span><span class="token operator">:</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">stringify</span><span class="token punctuation">(</span>items<span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">response</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> response<span class="token punctuation">.</span><span class="token function">json</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">catch</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">error</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span><span class="token string">'Error:'</span><span class="token punctuation">,</span> error<span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>Create an object array that will contain all the products that you want to add to the cart.</p>
<p><code>id</code> represents the variant ID.</p>
<p>You can also add subscription items if you also add a <code>selling_plan</code> ID which is created in Shopify when setting up a subscription and is required when adding a subscription product to the cart.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token keyword">let</span> products <span class="token operator">=</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'items'</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'id'</span><span class="token operator">:</span> <span class="token number">111</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'quantity'</span><span class="token operator">:</span> <span class="token number">4</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'id'</span><span class="token operator">:</span> <span class="token number">41618559008930</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'selling_plan'</span><span class="token operator">:</span> <span class="token number">222</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'quantity'</span><span class="token operator">:</span> <span class="token number">2</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">]</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>If needed, you can add another item to the <code>products</code> object by using <a href="https://www.w3schools.com/jsref/jsref_push.asp">push()</a>.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">products<span class="token punctuation">.</span>items<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token literal-property property">id</span><span class="token operator">:</span> <span class="token number">333</span><span class="token punctuation">,</span> <span class="token literal-property property">quantity</span><span class="token operator">:</span><span class="token number">5</span><span class="token punctuation">}</span><span class="token punctuation">)</span></span></code></pre>
<p>Which will add the product to the <code>products</code> object, in the <code>items</code> array which will be formatted correctly for Shopify's Cart API.</p>
<p>You can also use <a href="https://www.w3schools.com/jsref/jsref_pop.asp">pop()</a> to remove the last item in the array, or just destroy and recreate the <code>products</code> object if you want to start over again.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">products<span class="token punctuation">.</span>items<span class="token punctuation">.</span><span class="token function">pop</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span></code></pre>
<h2>Adding the products to the cart</h2>
<p>Once you have the object all ready, adding the items to the cart is a simple call to the function that we made earlier.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">Shopify<span class="token punctuation">.</span><span class="token function">addToCart</span><span class="token punctuation">(</span>products<span class="token punctuation">)</span></span></code></pre>
<p>There is more functionality available in the Shopify documentation for <a href="https://shopify.dev/api/ajax/reference/cart">Cart API</a>, such as:</p>
<ul>
<li>adding item properties</li>
<li>changing specific line items via index or line item keys</li>
<li>changing selling_plans</li>
</ul>
<p>I think there are some really interesting applications that this can be used for that are not prevalent in common shopping experiences today.</p>
Setting up Google Adwords Enhanced Conversions in Shopify2022-05-02T00:00:00Zhttps://www.robkjohnson.com/posts/setup-google-adwords-enhanced-conversions-shopify/<p>Google Adwords Enhanced Conversions provide more data to Google that help drive their algorithms to better optimize when and how your product listings and campaigns are shown to potential customers.</p>
<p>Below are instructions on the Enhanced Conversion script that is run after the checkout has been completed.</p>
<h3>Create a new Conversion in your Google Adwords account with Enhanced Conversions turned on</h3>
<p>The below image outlines how you want to setup a Coversion with Enhanced Conversions tracking.</p>
<p>You'll want to make sure you add the tracking via <code>Global site tag</code> which will allow you implement the tag on the back-end instead of Google Tag Manager which can be blocked with Ad-Blockers.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/google-adwords-conversion-setup.png" alt="Setup for a Google Enhanced Conversion" /></p>
<h3>Download Google Tag Assistant Chrome browser extension</h3>
<p>To help confirm that your tags are properly setup and verify the conversion labels are correct download Google Chrome extension <a href="https://chrome.google.com/webstore/detail/tag-assistant-legacy-by-g/kejbdjndbnbjgmefkgdddjlbokphdefk">Tag Assistant Legacy (by Google)</a>. You will use it to verify you have the correct tags firing on the correct pages.</p>
<h3>Shopify Config for Google Enhanced Conversions</h3>
<ol>
<li>Setup a conversion in your AdWords account</li>
<li>Verify that you have the correct conversion label</li>
<li>Verify that you have the correct Google Ads account number</li>
<li>Go to Shopify Admin > Settings > Checkout</li>
<li>Paste the below code in the <code>Order status page</code> section. (Make sure to replace <code>ACCOUNT_ID_HERE</code> and <code>CONVERSION_LABEL_HERE</code> with your Google Adwords account and conversion label.)</li>
</ol>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token operator"><</span>script<span class="token operator">></span></span>
<span class="highlight-line"><span class="token keyword">var</span> enhanced_conversion_data <span class="token operator">=</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">"first_name"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.first_name }}"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">"last_name"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.last_name }}"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">"home_address"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">"street"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.street }}"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">"city"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.city }}"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">"region"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.province }}"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">"postal_code"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.zip }}"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">"country"</span><span class="token operator">:</span> <span class="token string">"{{ checkout.billing_address.country_code }}"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token keyword">if</span><span class="token punctuation">(</span><span class="token string">"{{ checkout.email }}"</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> enhanced_conversion_data<span class="token punctuation">.</span>email <span class="token operator">=</span> <span class="token string">"{{ checkout.email }}"</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token keyword">if</span><span class="token punctuation">(</span><span class="token string">"{{ checkout.billing_address.phone }}"</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> enhanced_conversion_data<span class="token punctuation">.</span>phone_number <span class="token operator">=</span> <span class="token string">"{{ checkout.billing_address.phone }}"</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token operator"><</span><span class="token operator">/</span>script<span class="token operator">></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// Global site tag (gtag.js) - Google Ads: ACCOUNT_ID_HERE</span></span>
<span class="highlight-line"><span class="token operator"><</span>script <span class="token keyword">async</span> src<span class="token operator">=</span><span class="token string">"https://www.googletagmanager.com/gtag/js?id=AW-ACCOUNT_ID_HERE"</span><span class="token operator">></span><span class="token operator"><</span><span class="token operator">/</span>script<span class="token operator">></span></span>
<span class="highlight-line"><span class="token operator"><</span>script<span class="token operator">></span></span>
<span class="highlight-line"> window<span class="token punctuation">.</span>dataLayer <span class="token operator">=</span> window<span class="token punctuation">.</span>dataLayer <span class="token operator">||</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token keyword">function</span> <span class="token function">gtag</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span>dataLayer<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span>arguments<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token function">gtag</span><span class="token punctuation">(</span><span class="token string">'js'</span><span class="token punctuation">,</span> <span class="token keyword">new</span> <span class="token class-name">Date</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token function">gtag</span><span class="token punctuation">(</span><span class="token string">'config'</span><span class="token punctuation">,</span> <span class="token string">'AW-ACCOUNT_ID_HERE'</span><span class="token punctuation">,</span> <span class="token punctuation">{</span><span class="token string-property property">'allow_enhanced_conversions'</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token operator"><</span><span class="token operator">/</span>script<span class="token operator">></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token punctuation">{</span><span class="token operator">%</span> <span class="token keyword">if</span> first_time_accessed <span class="token operator">%</span><span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token operator"><</span>script<span class="token operator">></span></span>
<span class="highlight-line"> <span class="token function">gtag</span><span class="token punctuation">(</span><span class="token string">'event'</span><span class="token punctuation">,</span> <span class="token string">'conversion'</span><span class="token punctuation">,</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment">// add your account ID and Conversion label here</span></span>
<span class="highlight-line"> <span class="token string-property property">'send_to'</span><span class="token operator">:</span> <span class="token string">'AW-ACCOUNT_ID_HERE/CONVERSION_LABEL_HERE'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'value'</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token punctuation">{</span> checkout<span class="token punctuation">.</span>total_price <span class="token operator">|</span> divided_by<span class="token operator">:</span> <span class="token number">100.0</span> <span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'currency'</span><span class="token operator">:</span> <span class="token string">'{{ currency }}'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'transaction_id'</span><span class="token operator">:</span> <span class="token string">'{{ order_number }}'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token operator"><</span><span class="token operator">/</span>script<span class="token operator">></span></span>
<span class="highlight-line"><span class="token punctuation">{</span><span class="token operator">%</span> endif <span class="token operator">%</span><span class="token punctuation">}</span></span></code></pre>
<p>See also the <a href="https://support.google.com/google-ads/answer/9888145?hl=en_US">Google Support developer documentation</a></p>
How to minify, complile and output your sass/scss to a specific file location2022-04-29T00:00:00Zhttps://www.robkjohnson.com/posts/how-to-set-up-sass-scss-compiling-in-vs-code/<p>Using Sass or Scss is a great way to simplify the way you write css. However it requires compiling to css in order for it to be readable in a browser.</p>
<p>Here's how to set up automatic minification, compilation and output of the css file to a desired path.</p>
<p><strong>Requirement</strong>: This assumes you're using <a href="https://code.visualstudio.com/">VSCode</a>.</p>
<h3>Steps</h3>
<ol>
<li>Install the <a href="https://github.com/ritwickdey/vscode-live-sass-compiler">VSCode Live Sass Compiler</a> extension by selecting <code>View + Extensions</code> in the main menu and searching for <code>Live Sass Compiler</code>.</li>
<li>Create a folder <code>.vscode</code> at the root of your project if it doesn't already exist
<ul>
<li>Create a file <code>settings.json</code> in the new <code>.vscode</code></li>
</ul>
</li>
<li>Paste the following settings into your <code>.vscode/settings.json</code> file</li>
</ol>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"liveSassCompile.settings.formats"</span><span class="token operator">:</span> <span class="token punctuation">[</span></span>
<span class="highlight-line"> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment">// compress the css file</span></span>
<span class="highlight-line"> <span class="token property">"format"</span><span class="token operator">:</span> <span class="token string">"compressed"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token comment">// add .min.css as the extension</span></span>
<span class="highlight-line"> <span class="token property">"extensionName"</span><span class="token operator">:</span> <span class="token string">".min.css"</span><span class="token punctuation">,</span> </span>
<span class="highlight-line"> <span class="token comment">// define the output/save path</span></span>
<span class="highlight-line"> <span class="token property">"savePath"</span><span class="token operator">:</span> <span class="token string">"~/../_site/assets/styles/"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">]</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"liveSassCompile.settings.excludeList"</span><span class="token operator">:</span> <span class="token punctuation">[</span></span>
<span class="highlight-line"> <span class="token string">"/_styles/!screen.sass"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string">"**/node_modules/**"</span></span>
<span class="highlight-line"> <span class="token punctuation">]</span><span class="token punctuation">,</span> </span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Additional settings and config are <a href="https://github.com/ritwickdey/vscode-live-sass-compiler/blob/master/docs/settings.md">documented here</a> in the Github repo.</p>
Increase AOV with multi-product add-to-cart permalinks for your Shopify store2022-04-27T00:00:00Zhttps://www.robkjohnson.com/posts/shopify-add-multple-items-to-cart-with-one-link/<div class="callout">
<span class="is-info"><strong>Update 2023-9-18: </strong></span> Shopify have updated their permalink API. <a href="https://www.robkjohnson.com/posts/shopify-cart-permalinks-guide-selling-plans" title="See the updated article for Shopify permalinks">See the latest »</a>
</div>
<p>Running on Shopify and want a quick way to increase AOV?</p>
<p>Here are two permalink schemes that will allow you add mutliple variants and quantites to a cart.</p>
<p>You can use these custom permalinks anywhere. Custom theme templates, emails, social posts and so forth. You can dynamically create a bundle of products that get added to the cart all at once with no additional setup.</p>
<p>There are two URL schemes.</p>
<h3>Add order lines</h3>
<p>Add a product to the cart or, if the product already exists in the cart, increment the quantity of the product on the cart order line.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/cart/add?items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>id<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">123</span><span class="token operator">&</span>items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>quantity<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">1</span><span class="token operator">&</span>items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>id<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">123</span><span class="token operator">&</span>items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>quantity<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">3</span></span></code></pre>
<p>The numbers <code>123</code> and <code>456</code> represent <a href="https://help.shopify.com/en/manual/products/variants/find-variant-id">Shopify variant ID's</a> which can be found in the Shopify Admin URL when viewing the variant edit page.</p>
<p>See the Shopify URL path below.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/admin/products/11111/variants/88888</span></code></pre>
<p>In the above example, <code>11111</code> is the product ID and <code>88888</code> is the variant ID.</p>
<h3>Update order lines</h3>
<p>Update quantity for specific variants in the cart or, if the product already exists in the cart, update the quantity for that order line.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/cart/update?updates<span class="token punctuation">[</span><span class="token number">12345678910</span><span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">10</span><span class="token operator">&</span>updates<span class="token punctuation">[</span><span class="token number">12345678911</span><span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">2</span></span></code></pre>
<p>In the above example, <code>[12345678910]=10</code> represents the variant id <code>12345678910</code> and the quantity <code>10</code>.</p>
<p><strong>Limitation:</strong> If you already have a quantity of 10 for a given variant, the update method override the current quantity in the cart, with the quantity specified in the URL.</p>
<h3>A quick overview of Products vs Variants vs SKU vs Order Lines</h3>
<p>Both of these schemes rely on variant ID's. Variants are children of Products and have a "one-to-many" relationship. One product, to many variants.</p>
<p>For example a product called, "Super Cool T-Shirt" will can have many variants. These variant options can be color and size.</p>
<ul>
<li><strong>Color</strong>: Red, Blue, Green</li>
<li><strong>Size</strong>: S, M, L, XL, XXL</li>
</ul>
<p>A variant represents a single item or SKU (Stock Keeping Unit) and can be dislayed as <code>Super Cool T-Shirt - Gray L</code> to the customer, <code>Variant ID</code> to the ecommerce system and <code>SKU</code> as a business key (accounting, business intelligence etc).</p>
<p>We want to determine exactly what should be added to the cart, so we use Shopify's <code>variant ID</code> which is a Shopify system key.</p>
<p>There is a 1:1 relationship between a variant and a SKU in most ecommerce setups. You can have what is called a KIT SKU, which is a parent SKU that has many children SKUS.</p>
<p>Order lines have their own system keys called <code>order_line_key</code> or something to that effect. Because each order line can be unique due to taxes, discounts and such, they are recorded in an <code>order_line</code> table and are children to the <code>order_header</code> table which stores all of the summary information for an order. Usually each <code>order_line</code> has an <code>order_id</code> which can be used to join to the <code>order_header</code> table.</p>
<p>Shopify now have official documentation on <a href="https://shopify.dev/docs/apps/checkout/cart-permalinks/cart-permalinks">Creating Cart Permalinks</a>. It's also recommended that this approach replace Shopify's "Buy Button" feature.</p>
<h3>Adding a subscription with a selling plan</h3>
<p>If you're selling subscriptions on Shopify, you'll need to pass in the selling plan id associated with the variant also.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/cart/add?items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>id<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">123456789</span><span class="token operator">&</span>items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>quantity<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">1</span><span class="token operator">&</span>items<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">[</span>selling_plan<span class="token punctuation">]</span><span class="token operator">=</span><span class="token number">555</span></span></code></pre>
<p>You can also add <code>&return_to=/checkout</code> to send customers straight to the checkout.</p>
<p>If you need to clear the cart, you can hit the endpoint <code>/cart/clear?</code> instead with a <code>return_to=</code> param structured in the following way:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/cart/clear?return_to<span class="token operator">=</span>/cart/add%3Fitems%5B%5D%5Bid%5D%3D123456789%26items%5B%5D%5Bquantity%5D%3D1%26items%5B%5D%5Bselling_plan%5D%3D555%26return_to<span class="token operator">=</span>%2Fcheckout</span></code></pre>
<p><strong>Note</strong>: we need to encode the URL after the <code>?</code> so that the browser doesn't confuse params in the <code>return_to</code> URL with the first request of <code>/cart/clear</code>. Technically we are having Shopify run through three consecutive calls in one URL here. The first is <code>/cart/clear</code>, the second <code>/cart/add/</code> and the third <code>/checkout</code>.</p>
<p>If you have a discount you want to automatically add to a cart, while also adding items to the cart, use the following URL:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/discount/DISCOUNTCODEHERE?redirect<span class="token operator">=</span>/cart/add%3Fitems%5B%5D%5Bid%5D%3D123456789%26items%5B%5D%5Bquantity%5D%3D1%26items%5B%5D%5Bselling_plan%5D%3D555</span></code></pre>
<p>So we're applying a discount to the cart, then adding an item to the cart. By default the customer will land on the <code>/cart</code> page. If you'd like to redirect them somewhere else, like <code>/checkout</code> for example, add this at the end of the URL: <code>%26return_to=/checkout</code></p>
<h3>Add discount > add items to cart > go straight to checkout</h3>
<p>Full URL example:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/discount/DISCOUNTCODEHERE?redirect<span class="token operator">=</span>/cart/add%3Fitems%5B%5D%5Bid%5D%3D123456789%26items%5B%5D%5Bquantity%5D%3D1%26items%5B%5D%5Bselling_plan%5D%3D555%26return_to<span class="token operator">=</span>%2Fcheckout</span></code></pre>
<h3>AJAX API</h3>
<p>You can also pass a json object through to the the cart page via <code>/cart/add.js</code>.</p>
<p>This will allow you to increment quantity along with selling plans.</p>
<p>You can read more about this method on my post <a href="https://www.robkjohnson.com/posts/shopify-add-to-cart-mulitple-products-selling-plans-via-javascript-cart-api/">Add multiple products, subscriptions and qty to a Shopify cart, via Javascript Cart API</a></p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token function">fetch</span><span class="token punctuation">(</span><span class="token string">'/cart/add.js'</span><span class="token punctuation">,</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token literal-property property">method</span><span class="token operator">:</span> <span class="token string">"post"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">headers</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token string-property property">'content-type'</span><span class="token operator">:</span> <span class="token string">'application/json'</span> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">body</span><span class="token operator">:</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">stringify</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token literal-property property">items</span><span class="token operator">:</span> <span class="token punctuation">[</span></span>
<span class="highlight-line"> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token literal-property property">id</span><span class="token operator">:</span> <span class="token number">123456789</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">quantity</span><span class="token operator">:</span> <span class="token number">5</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token literal-property property">id</span><span class="token operator">:</span> <span class="token number">987654321</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">selling_plan</span><span class="token operator">:</span> <span class="token number">222</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">quantity</span><span class="token operator">:</span> <span class="token number">3</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">]</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span></span></code></pre>
Improve your product requirements process with Behavior-Driven Development & gherkin2022-04-21T00:00:00Zhttps://www.robkjohnson.com/posts/writing-requirements-with-gherkin-behavior-driven-development/<p>Behavior-driven development is very effective at distilling complex feature requests down to the essentials and can often help clear up misconceptions about processes, or desired features of a complex system.</p>
<p>It provides an approach of writing user stories in pseudocode which provides clarity to both the stakeholders and the developers.</p>
<p>I've been in situations where I've had several members of the same team around the table, discussing a current process each of them are involved in.</p>
<p>Writing in this style made clear that each of team members had different opinions of <em>what</em> and <em>how</em> the process should be performed.</p>
<p>This approach helped us to get a clear picture of what the requirements should be, whereas without this clarity, there could have been expensive additional iterations and corrections to the software which frustrate both the stakeholder and the developer.</p>
<p>Here is an example of a feature written in BDD using <a href="https://cucumber.io/docs/gherkin/reference/">Gherkin syntax</a>.</p>
<pre class="language-gherkin"><code class="language-gherkin"><span class="highlight-line"><span class="token feature"><span class="token keyword">Feature:</span></span></span>
<span class="highlight-line">Give new customers a free trial when signing up for a subscription</span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token scenario"><span class="token keyword">Scenario:</span><span class="token important"></span></span></span>
<span class="highlight-line">A customer signs up for a subscription that costs $5 per month which includes a 14-day free trial for new subscribers</span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token atrule">Given</span> <span class="token atrule">a</span> customer creates a new subscription and has not subscribed before</span>
<span class="highlight-line"> <span class="token atrule">When</span> they finish checkout, their subscription service starts immediately</span>
<span class="highlight-line"> <span class="token atrule">And</span> they are not charged for the subscription</span>
<span class="highlight-line"> <span class="token atrule">Then</span> after 14 days, they are automatically charged $5</span>
<span class="highlight-line"> <span class="token atrule">And</span> <span class="token atrule">a</span> new <span class="token string">"next_charge_scheduled_at"</span> date is set for the same day of the month in the following month</span>
<span class="highlight-line"> <span class="token atrule">Then</span> each renewal updates the <span class="token string">"next_charge_scheduled_at"</span> date</span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token atrule">Given</span> <span class="token atrule">a</span> customer cancels their subscription</span>
<span class="highlight-line"> <span class="token atrule">Then</span> the subscription status is set to <span class="token string">"canceled"</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment"># If a customer is past the free trial period</span></span>
<span class="highlight-line"> <span class="token atrule">When</span> the subscription has passed the free trial period</span>
<span class="highlight-line"> <span class="token atrule">Then</span> the membership service is set to expire when the current payment cycle ends</span>
<span class="highlight-line"> <span class="token atrule">And</span> the customer is not refunded for any previous payments (by default)</span>
<span class="highlight-line"> </span>
<span class="highlight-line"> <span class="token comment"># If a customer is still within the free trial period</span></span>
<span class="highlight-line"> <span class="token atrule">When</span> the subscription has not passed the free trial period</span>
<span class="highlight-line"> <span class="token atrule">Then</span> their service expires after the free trial period ends</span>
<span class="highlight-line"> <span class="token atrule">And</span> the customer is not charged for any subsequent renewals</span></code></pre>
<p>Even if I'm the only one writing the software, this approach can save countless hours of pointless coding by distilling down the core functionality while reviewing "bugs" in my own logic.</p>
<p><a href="https://cucumber.io/docs/bdd/">Learn more about BDD</a></p>
Send an HTTP request asynchronously with javascript & return the response2022-04-18T00:00:00Zhttps://www.robkjohnson.com/posts/send-http-request-asynchronously-with-javascript/<p>Hey, looks like you're accessing this page from IP <code id="your-ip">waiting...</code></p>
<p>Here's some javascript that reaches out to a free IP lookup service called, <a href="https://www.ipify.org/">ipify.org</a> (open source). The code execution is done asynchronously, which means that the javascript doesn't hold up the page from loading.</p>
<p>You can see this async call in action by reloading the page quickly and seeing the IP address above flicker with the text <code>waiting...</code>. You can also disable javascript in your browser to see the message.</p>
<p>Async calls are a good way to speed up the page if your page isn't dependent on the javascript executing immediately.</p>
<p>In the below code we make the request asynchornously, get response (JSON), parse the JSON and load it into a <code>code</code> element with the ID of <code>your-ip</code>.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// create the new object</span></span>
<span class="highlight-line"><span class="token keyword">let</span> request <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">XMLHttpRequest</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token comment">// define the response variable to store the object with the JSON already parsed</span></span>
<span class="highlight-line"><span class="token keyword">let</span> response <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></span>
<span class="highlight-line"><span class="token comment">// make a request with json as the response format</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function">open</span><span class="token punctuation">(</span><span class="token string">'GET'</span><span class="token punctuation">,</span> <span class="token string">'https://api.ipify.org?format=json'</span><span class="token punctuation">,</span> <span class="token boolean">true</span><span class="token punctuation">)</span> <span class="token comment">// set true for asynchronous</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function">setRequestHeader</span><span class="token punctuation">(</span><span class="token string">'Accept'</span><span class="token punctuation">,</span> <span class="token string">'application/json'</span><span class="token punctuation">)</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function-variable function">onreadystatechange</span> <span class="token operator">=</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>readyState <span class="token operator">===</span> <span class="token number">4</span> <span class="token operator">&&</span> <span class="token keyword">this</span><span class="token punctuation">.</span>status <span class="token operator">===</span> <span class="token number">200</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment">// result will be json</span></span>
<span class="highlight-line"> <span class="token comment">// example: {"ip":"127.0.0.1"}</span></span>
<span class="highlight-line"> <span class="token comment">// parse the json via JSON.parse</span></span>
<span class="highlight-line"> response <span class="token operator">=</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">parse</span><span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>responseText<span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token comment">// insert the value of the key "ip"</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">'your-ip'</span><span class="token punctuation">)</span><span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> response<span class="token punctuation">.</span>ip</span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token comment">// send back the response object</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function">send</span><span class="token punctuation">(</span>response<span class="token punctuation">)</span></span></code></pre>
<p>On line 11 of the code above we're requesting a <code>this.readyState</code> of <code>4</code> which means that the request is finished and response is ready.</p>
<p>We're also making sure that the <code>this.status</code> is <code>200</code> which means that the request has succesfully completed.</p>
<h3>XMLHttpRequest.readyState</h3>
<p><code>this.readyState</code> holds the status of the <code>XMLHttpRequest</code> which has the following responses:</p>
<table>
<thead>
<tr>
<th>Code</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>0</code></td>
<td>Request not initialized</td>
</tr>
<tr>
<td><code>1</code></td>
<td>Server connection established</td>
</tr>
<tr>
<td><code>2</code></td>
<td>Request received</td>
</tr>
<tr>
<td><code>3</code></td>
<td>Processing request</td>
</tr>
<tr>
<td><code>4</code></td>
<td>Request finished and response is ready</td>
</tr>
</tbody>
</table>
<p><a href="https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/readyState">Learn more about XMLHttpRequest.readyState</a></p>
<h3>HTML response status codes</h3>
<p>Every HTTP request has an HTML response status code that provides information about the response.</p>
<table>
<thead>
<tr>
<th>Code Range</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>100</code>-<code>199</code></td>
<td>Informational responses</td>
</tr>
<tr>
<td><code>200</code>–<code>299</code></td>
<td>Successful responses</td>
</tr>
<tr>
<td><code>300</code>–<code>399</code></td>
<td>Redirection messages</td>
</tr>
<tr>
<td><code>400</code>–<code>499</code></td>
<td>Client error responses</td>
</tr>
<tr>
<td><code>500</code>–<code>599</code></td>
<td>Server error responses</td>
</tr>
</tbody>
</table>
<p><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Status">Learn more about HTML response status codes</a></p>
<hr />
<h3>Going a step further</h3>
<p><a href="https://www.ipify.org/">ipify.org</a> is a good free solution to get the IP address only, however there are services that offer much more information.</p>
<p><a href="https://ipdata.co/">ipdata</a> offers an API that provides a lot of information from an IP. Here's an example of the response:</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"ip"</span><span class="token operator">:</span> <span class="token string">"127.0.0.1"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_eu"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"city"</span><span class="token operator">:</span> <span class="token string">"Seattle"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"region"</span><span class="token operator">:</span> <span class="token string">"Washington"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"region_code"</span><span class="token operator">:</span> <span class="token string">"WA"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"country_name"</span><span class="token operator">:</span> <span class="token string">"United States"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"country_code"</span><span class="token operator">:</span> <span class="token string">"US"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"continent_name"</span><span class="token operator">:</span> <span class="token string">"North America"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"continent_code"</span><span class="token operator">:</span> <span class="token string">"NA"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"latitude"</span><span class="token operator">:</span> <span class="token number">47.6080</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"longitude"</span><span class="token operator">:</span> <span class="token number">-122.3351</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"postal"</span><span class="token operator">:</span> <span class="token string">"98101"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"calling_code"</span><span class="token operator">:</span> <span class="token string">"1"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"flag"</span><span class="token operator">:</span> <span class="token string">"https://ipdata.co/flags/us.png"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"emoji_flag"</span><span class="token operator">:</span> <span class="token string">"\ud83c\uddfa\ud83c\uddf8"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"emoji_unicode"</span><span class="token operator">:</span> <span class="token string">"U+1F1FA U+1F1F8"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"asn"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"asn"</span><span class="token operator">:</span> <span class="token string">"AS33650"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"Comcast Cable Communications, LLC"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"domain"</span><span class="token operator">:</span> <span class="token string">"xfinity.com"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"route"</span><span class="token operator">:</span> <span class="token string">"127.0.0.0/17"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"type"</span><span class="token operator">:</span> <span class="token string">"isp"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"company"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"Comcast Cable Communications, Inc. "</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"domain"</span><span class="token operator">:</span> <span class="token string">"xfinity.com"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"type"</span><span class="token operator">:</span> <span class="token string">"isp"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"languages"</span><span class="token operator">:</span> <span class="token punctuation">[</span></span>
<span class="highlight-line"> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"English"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"native"</span><span class="token operator">:</span> <span class="token string">"English"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"code"</span><span class="token operator">:</span> <span class="token string">"en"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">]</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"currency"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"US Dollar"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"code"</span><span class="token operator">:</span> <span class="token string">"USD"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"symbol"</span><span class="token operator">:</span> <span class="token string">"$"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"native"</span><span class="token operator">:</span> <span class="token string">"$"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"plural"</span><span class="token operator">:</span> <span class="token string">"US dollars"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"time_zone"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"America/Los_Angeles"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"abbr"</span><span class="token operator">:</span> <span class="token string">"PDT"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"offset"</span><span class="token operator">:</span> <span class="token string">"-0700"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_dst"</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"current_time"</span><span class="token operator">:</span> <span class="token string">"2022-04-19T02:44:50-07:00"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"threat"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"is_tor"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_icloud_relay"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_proxy"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_datacenter"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_anonymous"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_known_attacker"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_known_abuser"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_threat"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"is_bogon"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"blocklists"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"count"</span><span class="token operator">:</span> <span class="token string">"4"</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>You can <a href="https://dashboard.ipdata.co/sign-up.html">sign up for free</a> and get <a href="https://ipdata.co/pricing.html">1500 calls per day</a> for non-commercial use. Pricing tiers grant access to commercial blocklists and VPN detection.</p>
<p>Here's a description of all of the <a href="https://docs.ipdata.co/api-reference/response-fields">response fields</a>.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token keyword">let</span> request <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">XMLHttpRequest</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token keyword">let</span> response <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></span>
<span class="highlight-line"></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function">open</span><span class="token punctuation">(</span><span class="token string">'GET'</span><span class="token punctuation">,</span> <span class="token string">'https://api.ipdata.co/?api-key=your-api-key-here'</span><span class="token punctuation">)</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function">setRequestHeader</span><span class="token punctuation">(</span><span class="token string">'Accept'</span><span class="token punctuation">,</span> <span class="token string">'application/json'</span><span class="token punctuation">)</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function-variable function">onreadystatechange</span> <span class="token operator">=</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>readyState <span class="token operator">===</span> <span class="token number">4</span> <span class="token operator">&&</span> <span class="token keyword">this</span><span class="token punctuation">.</span>status <span class="token operator">==</span> <span class="token number">200</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> response <span class="token operator">=</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">parse</span><span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>responseText<span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span><span class="token function">send</span><span class="token punctuation">(</span>response<span class="token punctuation">)</span></span></code></pre>
<p>These values can then be accessed by <code>response.ip</code> or <code>response.company.name</code> for example.</p>
<p><a class="button" id="ipdata-button" onclick="getIpData()" href="javascript:;">Click for a live demo</a></p>
<pre id="ipdata-response"></pre>
<script>
//--------------------------------------------------------------------------------------------------------------------//
// ipify
//--------------------------------------------------------------------------------------------------------------------//
let request = new XMLHttpRequest()
let response = []
request.open('GET', 'https://api.ipify.org?format=json', true)
request.setRequestHeader('Accept', 'application/json')
request.onreadystatechange = function () {
if (this.readyState === 4 && this.status === 200) {
response = JSON.parse(this.responseText)
document.getElementById('your-ip').innerHTML = response.ip
}
};
request.send(response)
//--------------------------------------------------------------------------------------------------------------------//
// ipdata
//--------------------------------------------------------------------------------------------------------------------//
function getIpData() {
let ipd_request = new XMLHttpRequest()
let ipd_response = []
ipd_request.open('GET', 'https://api.ipdata.co/?api-key=6651e53763a91fbbcc37df4d3d9c489d83473a53e6c175076d799465')
ipd_request.setRequestHeader('Accept', 'application/json')
ipd_request.onreadystatechange = function () {
if (this.readyState === 4 && this.status == 200) {
ipd_response = JSON.parse(this.responseText)
document.getElementById('ipdata-response').innerHTML = this.responseText
document.getElementById('ipdata-button').innerHTML = "Success: See response below"
document.getElementById('ipdata-response').className = "language-json"
document.getElementById('ipdata-button').className = "button is-success"
document.getElementById('ipdata-button').style.pointerEvents = "none"
} else if (this.status != 200) {
document.getElementById('ipdata-button').innerHTML = "Failed: " + this.status + " response. See message below"
document.getElementById('ipdata-response').innerHTML = "The request returned a " + this.status + " reponse: View the response codes above. It could be due to your current IP being on a threat list and rejected by ipdata.co. This can happen in you're using a VPN. Or that the 1500 request limit has been reached in the last 24 hours."
document.getElementById('ipdata-button').className = "button is-danger"
document.getElementById('ipdata-button').style.pointerEvents = "none"
}
};
ipd_request.send(ipd_response)
}
</script>
Sending a Shopify GraphQL query via HTTP request2022-04-15T00:00:00Zhttps://www.robkjohnson.com/posts/sending-shopify-graphql-query-via-http/<p>If you're using Shopify, you can query data for your store via several API's. One of them is the <a href="https://shopify.dev/api/admin-graphql">Shopify GraphQL Admin API</a> which is a great way to access only the data you need without multiple "hops" or API calls.</p>
<p>Shopify provide their own <a href="https://shopify.dev/apps/tools/graphiql-admin-api">GraphiQL app</a> that can be accessed via your stores admin. This is a great tool as it provides autocomplete which can help speed up the query building process and also help you learn the API faster.</p>
<p>Here's an example of a GraphQL query</p>
<pre class="language-graphql"><code class="language-graphql"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property-query">productVariant</span><span class="token punctuation">(</span><span class="token attr-name">id</span><span class="token punctuation">:</span> <span class="token string">"gid://shopify/ProductVariant/12345678912345"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">sku</span></span>
<span class="highlight-line"> <span class="token property-query">sellingPlanGroups</span><span class="token punctuation">(</span><span class="token attr-name">first</span><span class="token punctuation">:</span> <span class="token number">5</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token object">edges</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token object">node</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">id</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Which will return a json object response of</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"data"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"productVariant"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"sku"</span><span class="token operator">:</span> <span class="token string">"sku123456"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"sellingPlanGroups"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"edges"</span><span class="token operator">:</span> <span class="token punctuation">[</span></span>
<span class="highlight-line"> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"node"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"id"</span><span class="token operator">:</span> <span class="token string">"gid://shopify/SellingPlanGroup/987654321"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">]</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"extensions"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"cost"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"requestedQueryCost"</span><span class="token operator">:</span> <span class="token number">8</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"actualQueryCost"</span><span class="token operator">:</span> <span class="token number">4</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"throttleStatus"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"maximumAvailable"</span><span class="token operator">:</span> <span class="token number">2000</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"currentlyAvailable"</span><span class="token operator">:</span> <span class="token number">1996</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"restoreRate"</span><span class="token operator">:</span> <span class="token number">100</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>What's great here is that you also see the cost of the query which will help you measure the impact of the query if it's going to be used in a production/background job at a high frequency. Check out <a href="https://shopify.dev/api/usage/rate-limits">Shopify's API rate limits and leaky bucket algorithm that they use</a>.</p>
<p>But there's a good chance you may need to send a GraphQL query via an HTTP request from your app. Here's how you do that.</p>
<h2>Sending a GraphQL query via a standard HTTP request</h2>
<p>In this example we're going to send a mutation query, which will <a href="https://shopify.dev/api/admin-graphql/2022-01/mutations/fulfillmentOrderHold">change a Shopify fulfillment status to "ON_HOLD"</a>.</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"query"</span><span class="token operator">:</span> <span class="token string">"mutation ($fulfillmentHold: FulfillmentOrderHoldInput!, $id: ID!) { fulfillmentOrderHold(fulfillmentHold: $fulfillmentHold, id: $id) { fulfillmentOrder { id status } userErrors { field message } } }"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"variables"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"fulfillmentHold"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"notifyMerchant"</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"reason"</span><span class="token operator">:</span> <span class="token string">"OTHER"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"reasonNotes"</span><span class="token operator">:</span> <span class="token string">"Shopify Flow testing from Postman"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"id"</span><span class="token operator">:</span> <span class="token string">"gid://shopify/FulfillmentOrder/1234567891234"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Notice that the query is sent on one line in double quotes <code>""</code>. Variables are defined as separate objects.</p>
<p>That's it.</p>
How to prevent Javascript functions from running until the DOM is ready2022-04-15T00:00:00Zhttps://www.robkjohnson.com/posts/javascript-DOM-addEventListener-DOMContentLoaded/<p>DOM stands for Document Object Model which describes the overall contents and layout of an HTML document.</p>
<p>If you declare JavaScript that targets an HTML element before it has been rendered, you'll get an error.</p>
<p>A simple way to wait until all the DOM is ready and loaded is to use an <a href="https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener">EventListener</a> for the event <a href="https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event">DOMContentLoaded</a>.</p>
<p>This way you can execute whatever Javascript you want after the DOM has finished rendering.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'DOMContentLoaded'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment">// stuff you want to run here</span></span>
<span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">'DOM fully loaded and parsed'</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// show a message in the console</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
Overview of Javascript's URL() Constructor2022-04-01T00:00:00Zhttps://www.robkjohnson.com/posts/using-javascript-searchparams/<p>7 years ago, I wrote a <a href="https://www.robkjohnson.com/posts/javascript-dom-manipulation/">post about how to get params from a URL via javascript</a>. It used the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp">RegExp()</a> function, which isn't an elegant way, but worked fine to grab URL parameters.</p>
<p>Now there's a much better way to get, set, append and delete URL parameters in a URL.</p>
<h2>Javascripts URL() constructor</h2>
<p>You can get a breakdown of a URL's properties via the <code>URL()</code> constructor.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// get the current URL in the browser</span></span>
<span class="highlight-line"><span class="token comment">// and create a new URL() object</span></span>
<span class="highlight-line"><span class="token keyword">const</span> url <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">URL</span><span class="token punctuation">(</span>window<span class="token punctuation">.</span>location<span class="token punctuation">.</span>href<span class="token punctuation">)</span></span></code></pre>
<p>It will create an object with the following attributes.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token literal-property property">hash</span><span class="token operator">:</span> <span class="token string">""</span></span>
<span class="highlight-line"><span class="token literal-property property">host</span><span class="token operator">:</span> <span class="token string">"www.robkjohnson.com"</span></span>
<span class="highlight-line"><span class="token literal-property property">hostname</span><span class="token operator">:</span> <span class="token string">"www.robkjohnson.com"</span></span>
<span class="highlight-line"><span class="token literal-property property">href</span><span class="token operator">:</span> <span class="token string">"https://www.robkjohnson.com/"</span></span>
<span class="highlight-line"><span class="token literal-property property">origin</span><span class="token operator">:</span> <span class="token string">"https://www.robkjohnson.com"</span></span>
<span class="highlight-line"><span class="token literal-property property">password</span><span class="token operator">:</span> <span class="token string">""</span></span>
<span class="highlight-line"><span class="token literal-property property">pathname</span><span class="token operator">:</span> <span class="token string">"/"</span></span>
<span class="highlight-line"><span class="token literal-property property">port</span><span class="token operator">:</span> <span class="token string">""</span></span>
<span class="highlight-line"><span class="token literal-property property">protocol</span><span class="token operator">:</span> <span class="token string">"https:"</span></span>
<span class="highlight-line"><span class="token literal-property property">search</span><span class="token operator">:</span> <span class="token string">""</span></span>
<span class="highlight-line"><span class="token literal-property property">searchParams</span><span class="token operator">:</span> URLSearchParams <span class="token punctuation">{</span><span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token literal-property property">username</span><span class="token operator">:</span> <span class="token string">""</span></span></code></pre>
<p>One of the properties is called <code>searchParams</code> which is an array that will contain all of the parameters in the url that is passed into the <code>URL()</code> constructor.</p>
<p>You can use <code>searchParams</code> to set, get, append and delete url parameters as needed.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// define the variables/value for your params</span></span>
<span class="highlight-line"><span class="token keyword">const</span> size <span class="token operator">=</span> <span class="token string">'L'</span></span>
<span class="highlight-line"><span class="token keyword">const</span> color <span class="token operator">=</span> <span class="token string">'red'</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// set a param</span></span>
<span class="highlight-line">url<span class="token punctuation">.</span>searchParams<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">'size'</span><span class="token punctuation">,</span> size<span class="token punctuation">)</span> <span class="token comment">// adds ?size=L</span></span>
<span class="highlight-line">url<span class="token punctuation">.</span>searchParams<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">'color'</span><span class="token punctuation">,</span> color<span class="token punctuation">)</span> <span class="token comment">// adds &color=red</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// get a param</span></span>
<span class="highlight-line">url<span class="token punctuation">.</span>searchParams<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">'size'</span><span class="token punctuation">)</span> <span class="token comment">// L;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// append a param</span></span>
<span class="highlight-line">url<span class="token punctuation">.</span>searchParams<span class="token punctuation">.</span><span class="token function">append</span><span class="token punctuation">(</span><span class="token string">'fabric'</span><span class="token punctuation">,</span> <span class="token string">'cotton'</span><span class="token punctuation">)</span> <span class="token comment">// adds &fabric=cotton at the end of the url</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// delete a param</span></span>
<span class="highlight-line">url<span class="token punctuation">.</span>searchParams<span class="token punctuation">.</span><span class="token function">delete</span><span class="token punctuation">(</span><span class="token string">'size'</span><span class="token punctuation">)</span> <span class="token comment">// deletes the size parameter</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">// transform the url object to a string</span></span>
<span class="highlight-line">url<span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment">// https://www.robkjohnson.com/?color=red&fabric=cotton</span></span></code></pre>
How to use Liquid Filters in Shopify to Parse Global ID's2022-03-30T00:00:00Zhttps://www.robkjohnson.com/posts/using-shopify-liquid-filters-to-parse-global-id/<p>Shopify uses an open source language called, <a href="https://github.com/harttle/liquidjs">Liquid</a> for it's page and email templating and also logic within Shopify Flow. Liquid enables you to add objects/variables dynamically. It's helpful to learn about the syntax and Shopify has good documentation on what it supports:</p>
<ul>
<li><a href="https://shopify.dev/api/liquid">Shopify's Liquid API docs</a></li>
<li><a href="https://shopify.github.io/liquid/basics/introduction/">Shopify Liquid docs pn Github.io</a></li>
</ul>
<h2>Liquid Filter Example</h2>
<p>You want to place a HOLD on a <code>fulfillmentOrder</code> when an order contains more than 20 items. This will prevent the order from being automatically fulfilled via <a href="https://www.shipstation.com/">Shipstation</a>, or whatever 3PL provider you've connected to your store.</p>
<p>You've created a workflow in <a href="https://apps.shopify.com/flow">Shopify Flow</a> that will send an HTTP request once this condition is met. The problem is that the <code>fulfillmentOrders_item.id</code> variable returns as:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">gid://shopify/FulfillmentOrder/1234567890</span></code></pre>
<p>However, the <a href="https://shopify.dev/api/admin-rest/2021-10/resources/fulfillmentorder#post-fulfillment-orders-fulfillment-order-id-hold">endpoint</a> you're trying to hit requires only the integer of the <code>gid</code>.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">/admin/api/2021-10/fulfillment_orders/1234567890/hold.json</span></code></pre>
<p>So you need to somehow return only the number</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token number">1234567890</span></span></code></pre>
<p>This is where <a href="https://shopify.dev/api/liquid/filters">Liquid Filters</a> come in handy. You can use <code>split</code> and <code>last</code> in the following way to return <code>1234567890</code>.</p>
<pre class="language-liquid"><code class="language-liquid"><span class="highlight-line"><span class="token liquid language-liquid"><span class="token delimiter punctuation">{{</span>fulfillmentOrders_item<span class="token punctuation">.</span>id <span class="token operator">|</span> <span class="token function filter">split</span><span class="token operator">:</span> <span class="token string">"/"</span> <span class="token operator">|</span> <span class="token function filter">last</span><span class="token delimiter punctuation">}}</span></span></span></code></pre>
<p>The above filter splits on <code>/</code> and grabs the last split item which is <code>1234567890</code>.</p>
<p>There are a lot of applications for this in Shopify as each major object has a unique global identifyer assigned.</p>
Using Rollbar to catch 404 errors during an app migration2022-03-12T00:00:00Zhttps://www.robkjohnson.com/posts/using-rollbar-when-migrating-logging-404-errors/<p>Whenever you're migrating an app that is a number of years old, there will always be pages that you'll never catch. This includes links within product description pages (PDP's) or custom pages like landing pages, or customer support pages. There can be links pointing to your site from other search engines or sites that you and others don't even remember existing.</p>
<p>Some of these sound fairly trivial, however, you can also catch cases where you're not realizing how much traffic is being driven to a certain page, or group of pages, which is lowering your conversion rates and also search engine rankings.</p>
<h3>Google Search Console</h3>
<p>Google Search Console is great to show you the perspective of Google and what 404's they are detecting, however this isn't a fast enough response, in my opinion, especially when you're developing in a local, staging or beta environment behind public access.</p>
<p>In 2015. I wrote the post, <a href="https://www.robkjohnson.com/posts/5-mistakes-i-made-replatforming/">Top 5 Mistakes I Made Replatforming a Multimillion Dollar Ecommerce Website</a>. Point #4 was "Obsess over old external and internal links." Back then, my recommendation was to use <a href="https://www.screamingfrog.co.uk/seo-spider/">SEO Spider, a tool from Screaming Frog</a>. I still recommend this for a general crawl, especially if you're testing in a local or staging environment. However, it will only crawl links contained within the application and not links coming from external sites, which is why it's important to track 404's closely in your production environment.</p>
<p>A while ago, I wrote <a href="https://www.robkjohnson.com/posts/logging-with-rollbar/">How to use Rollbar for logging in a custom app</a>. Check it out if you're new to <a href="https://rollbar.com/">Rollbar</a> and it will help you understand more about the service and how to get started. They have an impressive set of SDK's that most likely cover your stack.</p>
<h3>Sending a warning log event via Rollbar's Javascript SDK</h3>
<p>With the most recent migration, I decided to add a custom log event anytime a 404 page was rendered on the site. Once Rollbar was setup, I just added the following Javascript to the 404 page.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">Rollbar<span class="token punctuation">.</span><span class="token function">warning</span><span class="token punctuation">(</span><span class="token string">"404 Error"</span><span class="token punctuation">,</span> <span class="token punctuation">{</span><span class="token literal-property property">referrerUrl</span><span class="token operator">:</span> document<span class="token punctuation">.</span>referrer<span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Adding, <code>{referrerUrl: document.referrer}</code> in the above will dynamically add a new data point to the error log. <code>document.referrer</code> will grab the origin of the referring URL.</p>
<p>Now anytime a 404 error occurs, I will receive a warning log event of that occurance in real-time. This of course, can be done in other ways for other types of events too. Let your imagination run wild.</p>
<p>That said, the default view in Rollbar will show you a 24 minutes, 60 hours and 60 days occurances chart, which is super helpful, but each occurance is listed out individually, which doesn't help you necessarily prioritize which URL's to fix first (if there are lots of errors occuring).</p>
<h3>Rollbar RSQL</h3>
<p>To help aggregate all of the errors together with a the count of occurances of the error, you can use a great tool that Rollbar have made called RSQL, which allows you to write SQL queries against the logs that have been collected.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span> </span>
<span class="highlight-line">request<span class="token punctuation">.</span>url</span>
<span class="highlight-line"><span class="token punctuation">,</span> <span class="token function">count</span><span class="token punctuation">(</span>request<span class="token punctuation">.</span>url<span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token keyword">FROM</span> item_occurrence</span>
<span class="highlight-line"><span class="token keyword">WHERE</span> item<span class="token punctuation">.</span>counter <span class="token operator">=</span> <span class="token number">67</span></span>
<span class="highlight-line"><span class="token operator">AND</span> request<span class="token punctuation">.</span>url <span class="token operator">NOT</span> <span class="token operator">LIKE</span> <span class="token string">'%url-fragment-to-ignore%'</span></span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span></span>
<span class="highlight-line">request<span class="token punctuation">.</span>url</span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span> <span class="token function">count</span><span class="token punctuation">(</span>request<span class="token punctuation">.</span>url<span class="token punctuation">)</span> <span class="token keyword">DESC</span></span>
<span class="highlight-line"><span class="token keyword">LIMIT</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">100</span></span></code></pre>
<p>The results will provide a chart, if the data output is simple enough, and a downloadable CSV file, where you can then upload to Excel for whatever needs you may have.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/rollbar-rsql.png" alt="Rollbar RSQL Output Chart" /></p>
<p class="caption">Rollbar RSQL output chart</p>
<p>There you go. Tracking 404's in real-time, where you can also <a href="https://docs.rollbar.com/docs/notifications">setup notifications for certain events</a> and more.</p>
Creating IF functionality with SQL using CASE & subqueries2021-08-30T00:00:00Zhttps://www.robkjohnson.com/posts/creating-if-statements-in-sql-with-case-and-having/<p>When you have an app that is a number of years old, you'll find that the database schema has changed over the years. As a DB admin or analyst, this can be very frustrating as you'll find that you have to write logic in your SQL to work around having data you need in two different places/tables, with the need to combine them into one column for analysis.</p>
<h2>Scenario</h2>
<p>You need to export all of your article and author data from one database to another. The problem is that author data is stored in two locations after a change was made to the database schema.</p>
<p>Initially the app stored authors within the <code>article</code> table as a string value in the <code>author</code> column.</p>
<div class="table">
<table>
<thead>
<tr>
<th>article_id</th>
<th>author</th>
<th>slug</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Mary Poppins</td>
<td>twas-an-east-wind</td>
</tr>
<tr>
<td>2</td>
<td>Jane Banks</td>
<td>my-dad-doesnt-listen</td>
</tr>
</tbody>
</table>
</div>
<p>After a while, a change was made to the app where two new tables were created, <code>authorships</code> and <code>author</code>. This allowed multiple authors to be connected to an article; a much better choice of schema. Now in the <code>authors</code> table we can have additional information about the author that can be used to populate an author page and have separate links on an article page and ensure we keep each author separate in the database.</p>
<div class="table">
<table>
<thead>
<tr>
<th>author_id</th>
<th>first_name</th>
<th>last_name</th>
<th>twitter</th>
</tr>
</thead>
<tbody>
<tr>
<td>110</td>
<td>Mary</td>
<td>Poppins</td>
<td>twitter.com/mpoppins</td>
</tr>
<tr>
<td>111</td>
<td>Jane</td>
<td>Banks</td>
<td>twitter.com/jbanks</td>
</tr>
</tbody>
</table>
</div>
<p>The <code>authorships</code> table can associate many articles to many authors. In the example below we have two authors <code>110</code> and <code>111</code> assigned to <code>article_id</code> <code>1</code>.</p>
<div class="table">
<table>
<thead>
<tr>
<th>author_id</th>
<th>article_id</th>
</tr>
</thead>
<tbody>
<tr>
<td>110</td>
<td>1</td>
</tr>
<tr>
<td>111</td>
<td>1</td>
</tr>
</tbody>
</table>
</div>
<p>The problem is that when the new tables were created, there wasn't a migration for the old data to the new tables. So some articles will have the author information stored in the <code>articles</code> table and other articles will have a <code>BLANK</code> value in the <code>articles</code> table and instead have information stored in the <code>authorships</code>/<code>authors</code> tables.</p>
<p>So you are now faced with three tables (<code>articles</code>, <code>authorships</code> and <code>authors</code>) that have unique information for the authorship of each article.</p>
<h3>Tech debt</h3>
<p>Why did the app developer not migrate the author data? The app requires logic to render the article page properly. It has to now check both tables for an author entry and use an IF statement to determine what table to use. Ideally the app developer would have migrated all old articles to use the new <code>authorships</code> and <code>author</code> tables, so that all article data is stored in the same way in the database, instead of creating a divide in how the author data is stored.</p>
<p>This is called "technical debt" and you need to pay up.</p>
<h2>IF statement to the rescue</h2>
<p>If you were to solve this with an IF statement with a scripting language like Python, you could do the following:</p>
<pre class="language-python"><code class="language-python"><span class="highlight-line"><span class="token comment"># check to see if the article table has an author</span></span>
<span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span> article_author <span class="token operator">!=</span> NULL <span class="token punctuation">)</span><span class="token punctuation">:</span></span>
<span class="highlight-line"> <span class="token comment"># get the author from the article table</span></span>
<span class="highlight-line"> author_byline <span class="token operator">=</span> article_author</span>
<span class="highlight-line"> <span class="token keyword">else</span><span class="token punctuation">:</span></span>
<span class="highlight-line"> <span class="token comment"># get the author from the authorships table</span></span>
<span class="highlight-line"> author_byline <span class="token operator">=</span> authorship_author</span></code></pre>
<p>This is straighforward and may be the first thing you think of doing. IF/ELSE is what computers were designed for. However when you're doing analysis, you'll want to avoid having to run your analysis with an additional scripting language like Python etc. We can <em>transform</em> the data on the fly using the right SQL operators.</p>
<h2>Creating an IF statement with SQL using <code>CASE</code></h2>
<p><code>CASE</code> is a great way to perform conditional logic in your SQL. You can throw it in your <code>SELECT</code> or <code>WHERE</code> statement. Here is an example.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> <span class="token keyword">CASE</span></span>
<span class="highlight-line"> <span class="token keyword">WHEN</span> author<span class="token punctuation">.</span>first_name <span class="token operator">=</span> <span class="token string">'Mary'</span> <span class="token operator">AND</span> author<span class="token punctuation">.</span>last_name <span class="token operator">=</span> <span class="token string">'Poppins'</span></span>
<span class="highlight-line"> <span class="token keyword">THEN</span> CONCAT<span class="token punctuation">(</span>author<span class="token punctuation">.</span>first_name<span class="token punctuation">,</span> <span class="token string">' '</span><span class="token punctuation">,</span> author<span class="token punctuation">.</span>last_name<span class="token punctuation">,</span> <span class="token string">' is practically perfect in every way'</span><span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">ELSE</span> CONCAT<span class="token punctuation">(</span>author<span class="token punctuation">.</span>first_name<span class="token punctuation">,</span> <span class="token string">' '</span><span class="token punctuation">,</span> author<span class="token punctuation">.</span>last_name<span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">END</span></span>
<span class="highlight-line"><span class="token keyword">FROM</span></span>
<span class="highlight-line">authors</span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
<p>Here we check the author name. <code>WHEN</code> author matches 'Mary Poppins' <code>THEN</code> we create a new value on the fly by using <a href="https://www.postgresqltutorial.com/postgresql-concat-function/">CONCAT()</a> to put together a string of text that appends ' is practically perfect in every way'. The end result being "Marry Poppins is practically perfect in every way".</p>
<p>Silly example, but you get the point.</p>
<h3>Writing the initial query</h3>
<p>I always like to write pseudo SQL in the beginning so I can annotate the information I need with any details that are helpful to remember. I also add a <code>LIMIT 100</code> (if using MySQL or Postgres) or <code>TOP 100</code> (if using MSSQL) to ensure I'm not burdening the database server. I do this as a habit to ensure I'm not needlessly grabbing 100,000's of rows for no reason. I just need to QA a small data set for now.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line">articles<span class="token punctuation">.</span>article_id</span>
<span class="highlight-line"><span class="token punctuation">,</span> articles<span class="token punctuation">.</span>article_url</span>
<span class="highlight-line"><span class="token comment">-- check for the author name from the articles table</span></span>
<span class="highlight-line"><span class="token punctuation">,</span> articles<span class="token punctuation">.</span>author</span>
<span class="highlight-line"><span class="token comment">-- check for the author name from the authorships/author table</span></span>
<span class="highlight-line"><span class="token punctuation">,</span> CONCAT<span class="token punctuation">(</span>authors<span class="token punctuation">.</span>first_name<span class="token punctuation">,</span> <span class="token string">' '</span><span class="token punctuation">,</span> authors<span class="token punctuation">.</span>last_name<span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token punctuation">,</span> articles<span class="token punctuation">.</span>article_status</span>
<span class="highlight-line"><span class="token keyword">FROM</span></span>
<span class="highlight-line">articles</span>
<span class="highlight-line"><span class="token keyword">LIMIT</span> <span class="token number">100</span></span></code></pre>
<p>This query is unusable. But it's a starting point to help me understand the data objectives I have.</p>
<p>You may notice that in order to get <code>authors.first_name</code> and <code>authors.last_name</code> you would need to <code>JOIN</code> two tables <code>authors</code> and <code>authorships</code>. That <em>could</em> be done with the following <code>JOIN</code> statements:</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">LEFT</span> <span class="token keyword">JOIN</span> authorships <span class="token keyword">ON</span> articles<span class="token punctuation">.</span>article_id <span class="token operator">=</span> authorships<span class="token punctuation">.</span>article_id</span>
<span class="highlight-line"><span class="token keyword">LEFT</span> <span class="token keyword">JOIN</span> authors <span class="token keyword">ON</span> authors<span class="token punctuation">.</span>author_id <span class="token operator">=</span> authorships<span class="token punctuation">.</span>author<span class="token punctuation">.</span>id</span></code></pre>
<p>I'd do a <code>LEFT JOIN</code> here, as I know that only a subset of articles have author information assigned to them. A <code>LEFT JOIN</code> ensures that I show all the article rows even if the <code>authorships</code> table doesn't have any data connected to an article. If I left it as a simple <code>JOIN</code> or <code>INNER JOIN</code> then the only results I'd get back would be articles that <em>do</em> have a connected author via the <code>authorships</code> table.</p>
<p>This all said though, we actually want to add this <code>JOIN</code> inside of a subquery.</p>
<h3>Writing an SQL Subquery</h3>
<p>A subquery is a regular query that is put inside of parenthesis. You can add a subquery in a <code>SELECT</code> statement, or in a <code>WHERE</code> statement. I personally like to write my subqueries separately and test them before adding them as a subquery. Helps with QA.</p>
<h3>Complete SQL query</h3>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line">article<span class="token punctuation">.</span>article_id</span>
<span class="highlight-line"><span class="token punctuation">,</span> article<span class="token punctuation">.</span>url</span>
<span class="highlight-line"><span class="token punctuation">,</span> <span class="token keyword">CASE</span></span>
<span class="highlight-line"> <span class="token keyword">WHEN</span> article<span class="token punctuation">.</span>author <span class="token operator">IS</span> <span class="token boolean">NULL</span></span>
<span class="highlight-line"> <span class="token keyword">THEN</span> <span class="token punctuation">(</span><span class="token keyword">SELECT</span> CONCAT<span class="token punctuation">(</span>authors<span class="token punctuation">.</span><span class="token keyword">first</span><span class="token punctuation">,</span> <span class="token string">' '</span><span class="token punctuation">,</span> authors<span class="token punctuation">.</span><span class="token keyword">last</span><span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">FROM</span> authors</span>
<span class="highlight-line"> <span class="token keyword">JOIN</span> authorships <span class="token keyword">ON</span> authorships<span class="token punctuation">.</span>author_id <span class="token operator">=</span> authors<span class="token punctuation">.</span>author_id</span>
<span class="highlight-line"> <span class="token keyword">WHERE</span> authorships<span class="token punctuation">.</span>article_id <span class="token operator">=</span> a<span class="token punctuation">.</span>article_id </span>
<span class="highlight-line"> <span class="token keyword">LIMIT</span> <span class="token number">1</span><span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">ELSE</span> article<span class="token punctuation">.</span>author</span>
<span class="highlight-line"> <span class="token keyword">END</span> <span class="token keyword">as</span> author</span>
<span class="highlight-line"><span class="token punctuation">,</span> articles<span class="token punctuation">.</span>article_status</span>
<span class="highlight-line"><span class="token keyword">FROM</span></span>
<span class="highlight-line">articles</span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
Create an AWS Lambda to Query Data with Athena & Output to S3 Bucket using Python2021-08-16T00:00:00Zhttps://www.robkjohnson.com/posts/using-aws-lambda-python-athena-to-etl-data/<p>Traditionally I've used servers to run ETL jobs. PHP specifically has been my main tool over the years. I've used PHP since my Wordpress days back in 2007 and I've enjoyed using it. Simple, readable and readily available across the web ecosystem.</p>
<p>However, as great and easy as it is to get rolling, when you're running production services, worrying about the infrastructure and dev ops side of things can catch you off-guard and ruin your day if there is a problem. Serverless services such as AWS Lambda promise to take away that pain and provide you a way to focus on the real thing you care about, the execution of the code you write.</p>
<p>I'm a sucker for simplicity. I'll work really hard to be really lazy.</p>
<h2>Why Amazon Athena?</h2>
<p>Databases cost a lot to run in both time and cash. If you don't need high availablity and speed for your database, you should consider storing your data in a storage solution like Amazon Athena. It will store your data in S3 at S3 prices and only charge you for the queries you execute (time + size of data queried). For example, if you need to process some data every day, you likely don't need a production server running all day. This is especially true if you are running large queries on large datasets. Instead you could have your data dropped off in an S3 bucket in CSV format and you could use Athena to query those files the same way you'd query a regular SQL database.</p>
<blockquote>
<p>Athena helps you analyze unstructured, semi-structured, and structured data stored in Amazon S3. Examples include CSV, JSON, or columnar data formats such as Apache Parquet and Apache ORC. You can use Athena to run ad-hoc queries using ANSI SQL, without the need to aggregate or load the data into Athena.</p>
</blockquote>
<p class="caption"><a href="https://docs.aws.amazon.com/athena/latest/ug/when-should-i-use-ate.html" title="Amazon Athena Overview">AWS Athena Docs</a></p>
<p>I like using Athena as I can use SQL which I'm comfortable using while benefiting from having unlimited storage in S3. I don't need to worry about maintaining and paying for a database. The permissions are set within AWS and it is locked by default down to the AWS console if all the defaults are kept.</p>
<p>If storage (AWS S3) and data querying (AWS Athena) are both serverless, then why not use AWS lambda to run the job?</p>
<h2>Lambda 1: Query Athena and load the results into S3 (Python)</h2>
<p>In the example below, the code instructs the Lambda to import <code>boto3</code> (the <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">AWS SDK for Python</a>) and use it to run a query against a database/table, then output the results of that query in CSV format and upload to a selected S3 bucket.</p>
<p>This example is taken from <a href="https://aws.amazon.com/premiumsupport/knowledge-center/schedule-query-athena/">this AWS knowledge center doc</a></p>
<pre class="language-python"><code class="language-python"><span class="highlight-line"><span class="token keyword">import</span> time</span>
<span class="highlight-line"><span class="token keyword">import</span> boto3</span>
<span class="highlight-line"></span>
<span class="highlight-line">query <span class="token operator">=</span> <span class="token string">'SELECT * FROM default.tb'</span></span>
<span class="highlight-line">DATABASE <span class="token operator">=</span> <span class="token string">'DATABASE-NAME-HERE'</span></span>
<span class="highlight-line">output<span class="token operator">=</span><span class="token string">'s3://AWSDOC-EXAMPLE-BUCKET/'</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">def</span> <span class="token function">lambda_handler</span><span class="token punctuation">(</span>event<span class="token punctuation">,</span> context<span class="token punctuation">)</span><span class="token punctuation">:</span></span>
<span class="highlight-line"> client <span class="token operator">=</span> boto3<span class="token punctuation">.</span>client<span class="token punctuation">(</span><span class="token string">'athena'</span><span class="token punctuation">)</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment"># Execution</span></span>
<span class="highlight-line"> response <span class="token operator">=</span> client<span class="token punctuation">.</span>start_query_execution<span class="token punctuation">(</span></span>
<span class="highlight-line"> QueryString<span class="token operator">=</span>query<span class="token punctuation">,</span></span>
<span class="highlight-line"> QueryExecutionContext<span class="token operator">=</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string">'Database'</span><span class="token punctuation">:</span> DATABASE</span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> ResultConfiguration<span class="token operator">=</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string">'OutputLocation'</span><span class="token punctuation">:</span> output<span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> response</span>
<span class="highlight-line"> <span class="token keyword">return</span></span></code></pre>
<p>In my case, I wanted to change the location of the S3 bucket to point to a directory within the bucket. So I added a <code>path</code> variable and appended the path to the end of the output location in the <code>lambda_handler</code></p>
<pre class="language-python"><code class="language-python"><span class="highlight-line"><span class="token keyword">import</span> time</span>
<span class="highlight-line"><span class="token keyword">import</span> boto3</span>
<span class="highlight-line"></span>
<span class="highlight-line">query <span class="token operator">=</span> <span class="token string">'SELECT * FROM default.tb'</span></span>
<span class="highlight-line">DATABASE <span class="token operator">=</span> <span class="token string">'DATABASE-NAME-HERE'</span></span>
<span class="highlight-line">output<span class="token operator">=</span><span class="token string">'s3://AWSDOC-EXAMPLE-BUCKET/'</span></span>
<span class="highlight-line">path<span class="token operator">=</span><span class="token string">'parent-directory-name/child-directory-name/etc'</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">def</span> <span class="token function">lambda_handler</span><span class="token punctuation">(</span>event<span class="token punctuation">,</span> context<span class="token punctuation">)</span><span class="token punctuation">:</span></span>
<span class="highlight-line"> client <span class="token operator">=</span> boto3<span class="token punctuation">.</span>client<span class="token punctuation">(</span><span class="token string">'athena'</span><span class="token punctuation">)</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment"># Execution</span></span>
<span class="highlight-line"> response <span class="token operator">=</span> client<span class="token punctuation">.</span>start_query_execution<span class="token punctuation">(</span></span>
<span class="highlight-line"> QueryString<span class="token operator">=</span>query<span class="token punctuation">,</span></span>
<span class="highlight-line"> QueryExecutionContext<span class="token operator">=</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string">'Database'</span><span class="token punctuation">:</span> DATABASE</span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> ResultConfiguration<span class="token operator">=</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment"># first {} contains the output variable, then adds a '/' character for the directory and then the path variable</span></span>
<span class="highlight-line"> <span class="token string">'OutputLocation'</span><span class="token punctuation">:</span> <span class="token string">"{}/{}"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>output<span class="token punctuation">,</span> path<span class="token punctuation">)</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> response</span>
<span class="highlight-line"> <span class="token keyword">return</span></span></code></pre>
<p>Now write the SQL you want to use. In this example, I'll query all of the email complaints from a Pinpoint application/project over the last 3 days.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> complained_email_address</span>
<span class="highlight-line"><span class="token keyword">FROM</span> complain_count</span>
<span class="highlight-line"><span class="token keyword">WHERE</span></span>
<span class="highlight-line"> ingest_timestamp <span class="token operator">>=</span> <span class="token function">NOW</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token keyword">interval</span> <span class="token string">'3'</span> <span class="token keyword">day</span></span>
<span class="highlight-line"> <span class="token operator">AND</span> application_id <span class="token operator">=</span> <span class="token string">'123456789'</span></span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span> complained_email_address</span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
<p>Now I can add that to the lambda code I wrote above and change the name of the database to my database name in Athena <code>pinpoint_events</code>. I'll also add the bucket name <code>pinpoint-event-data</code> as the AWS S3 bucket name with the directory path <code>application_1/complaints</code>.</p>
<pre class="language-python"><code class="language-python"><span class="highlight-line"><span class="token keyword">import</span> time</span>
<span class="highlight-line"><span class="token keyword">import</span> boto3</span>
<span class="highlight-line"></span>
<span class="highlight-line">query <span class="token operator">=</span> <span class="token string">'SELECT complained_email_address FROM complain_count WHERE ingest_timestamp >= NOW() - interval '</span><span class="token number">3</span><span class="token string">' day AND application_id = '</span><span class="token number">123456789</span><span class="token string">' GROUP BY complained_email_address;'</span></span>
<span class="highlight-line">DATABASE <span class="token operator">=</span> <span class="token string">'pinpoint_events'</span></span>
<span class="highlight-line">output<span class="token operator">=</span><span class="token string">'s3://pinpoint-events-data/'</span></span>
<span class="highlight-line">path<span class="token operator">=</span><span class="token string">'application_1/complaints'</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">def</span> <span class="token function">lambda_handler</span><span class="token punctuation">(</span>event<span class="token punctuation">,</span> context<span class="token punctuation">)</span><span class="token punctuation">:</span></span>
<span class="highlight-line"> client <span class="token operator">=</span> boto3<span class="token punctuation">.</span>client<span class="token punctuation">(</span><span class="token string">'athena'</span><span class="token punctuation">)</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment"># Execution</span></span>
<span class="highlight-line"> response <span class="token operator">=</span> client<span class="token punctuation">.</span>start_query_execution<span class="token punctuation">(</span></span>
<span class="highlight-line"> QueryString<span class="token operator">=</span>query<span class="token punctuation">,</span></span>
<span class="highlight-line"> QueryExecutionContext<span class="token operator">=</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string">'Database'</span><span class="token punctuation">:</span> DATABASE</span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> ResultConfiguration<span class="token operator">=</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment"># first {} contains the output variable, then adds a '/' character for the directory and then the path variable</span></span>
<span class="highlight-line"> <span class="token string">'OutputLocation'</span><span class="token punctuation">:</span> <span class="token string">"{}/{}"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>output<span class="token punctuation">,</span> path<span class="token punctuation">)</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> response</span>
<span class="highlight-line"> <span class="token keyword">return</span></span></code></pre>
<p>You can setup a test in AWS Lambda function and just pass in a junk json payload to initiate the function.</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"key1"</span><span class="token operator">:</span> <span class="token string">"value1"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"key2"</span><span class="token operator">:</span> <span class="token string">"value2"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"key3"</span><span class="token operator">:</span> <span class="token string">"value3"</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>It doesn't matter what is here, none of it will be used in the <code>lambda_handler</code> function. We're just wanting to invoke the function and we need to pass some json in the request to do that.</p>
<p>After clicking "Test" you should a response like the following:</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line">Response</span>
<span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"QueryExecutionId"</span><span class="token operator">:</span> <span class="token string">"123456789"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"ResponseMetadata"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"RequestId"</span><span class="token operator">:</span> <span class="token string">"987654321"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"HTTPStatusCode"</span><span class="token operator">:</span> <span class="token number">200</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"HTTPHeaders"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"content-type"</span><span class="token operator">:</span> <span class="token string">"application/x-amz-json-1.1"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"date"</span><span class="token operator">:</span> <span class="token string">"Wed, 18 Aug 2021 06:17:16 GMT"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"x-amzn-requestid"</span><span class="token operator">:</span> <span class="token string">"794613"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"content-length"</span><span class="token operator">:</span> <span class="token string">"59"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"connection"</span><span class="token operator">:</span> <span class="token string">"keep-alive"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token property">"RetryAttempts"</span><span class="token operator">:</span> <span class="token number">0</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The <code>HTTPStatusCode</code> shows you that the Lambda ran successfully. You could skip ahead and check to see if your S3 bucket now contains a new .csv file with the data you wanted. However, you may have an error in your query. <strong>This will not reflect in this response</strong>. All this response is saying is, "yeah, looks good to me and I've passed it over to Athena to execute." In order to see the details of the query execution, you'll need to grab the <code>QueryExecutionId</code> and run some python in a python shell to view the results.</p>
<h2>Debugging your Athena Query within your Lambda Function</h2>
<p>This is a little more painful than I wish it was. So definitely do all your query QA directly in the AWS Athena console first.</p>
<p>Initiate the Cloud Shell in AWS (this assumes your role has permissions setup for this). You can refer to my previous post on <a href="https://www.robkjohnson.com/posts/setup-an-interactive-python-shell-on-your-local-machine/">how to install an iPython shell</a> on your local machine (and AWS Cloud Shell).</p>
<p>Import <code>boto3</code></p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">import</span> boto3</span></code></pre>
<p>Grab the <code>QueryExecutionId</code> which in this example is <code>123456789</code> and assign it to a varaible, then setup a client and grab the <code>QueryExecution</code> response.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">execution_id <span class="token operator">=</span> <span class="token string">'123456789'</span></span>
<span class="highlight-line">client <span class="token operator">=</span> boto3.client<span class="token punctuation">(</span><span class="token string">'athena'</span><span class="token punctuation">)</span></span>
<span class="highlight-line">response <span class="token operator">=</span> client.get_query_execution<span class="token punctuation">(</span>QueryExecutionId<span class="token operator">=</span>execution_id<span class="token punctuation">)</span></span></code></pre>
<p>Here is an example of a response that had an <strong>error</strong> due to insufficient permissions.</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span>'QueryExecution'<span class="token operator">:</span> <span class="token punctuation">{</span>'QueryExecutionId'<span class="token operator">:</span> '<span class="token number">123456789</span>'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'Query'<span class="token operator">:</span> <span class="token string">"SELECT complained_email_address FROM complain_count WHERE ingest_timestamp >= NOW() - interval '3' day AND application_id = '123456789' GROUP BY complained_email_address;"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'StatementType'<span class="token operator">:</span> 'DML'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'ResultConfiguration'<span class="token operator">:</span> <span class="token punctuation">{</span>'OutputLocation'<span class="token operator">:</span> 's3<span class="token operator">:</span><span class="token comment">//pinpoint-events-data/application_1/complaints/baf0294e-d839-47b6-be58-ca14615e4794.csv'},</span></span>
<span class="highlight-line"> 'QueryExecutionContext'<span class="token operator">:</span> <span class="token punctuation">{</span>'Database'<span class="token operator">:</span> 'pinpoint_events-data'<span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'Status'<span class="token operator">:</span> <span class="token punctuation">{</span>'State'<span class="token operator">:</span> 'FAILED'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'StateChangeReason'<span class="token operator">:</span> 'Insufficient permissions to execute the query. User<span class="token operator">:</span> arn<span class="token operator">:</span>aws<span class="token operator">:</span>sts<span class="token operator">:</span><span class="token operator">:</span><span class="token number">123</span><span class="token operator">:</span>assumed-role/pinpoint_role/pinpoint_process_complaints_lambda_name is not authorized to perform<span class="token operator">:</span> glue<span class="token operator">:</span>GetPartitions on resource<span class="token operator">:</span> arn<span class="token operator">:</span>aws<span class="token operator">:</span>glue<span class="token operator">:</span>us-west<span class="token number">-2</span><span class="token operator">:</span><span class="token number">123</span><span class="token operator">:</span>catalog '<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'SubmissionDateTime'<span class="token operator">:</span> datetime.datetime(<span class="token number">2021</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">,</span> <span class="token number">23</span><span class="token punctuation">,</span> <span class="token number">29</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">,</span> <span class="token number">869000</span><span class="token punctuation">,</span> tzinfo=tzlocal())<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'CompletionDateTime'<span class="token operator">:</span> datetime.datetime(<span class="token number">2021</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">,</span> <span class="token number">23</span><span class="token punctuation">,</span> <span class="token number">29</span><span class="token punctuation">,</span> <span class="token number">17</span><span class="token punctuation">,</span> <span class="token number">584000</span><span class="token punctuation">,</span> tzinfo=tzlocal())<span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'Statistics'<span class="token operator">:</span> <span class="token punctuation">{</span>'EngineExecutionTimeInMillis'<span class="token operator">:</span> <span class="token number">10508</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'DataScannedInBytes'<span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'TotalExecutionTimeInMillis'<span class="token operator">:</span> <span class="token number">10715</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'QueryQueueTimeInMillis'<span class="token operator">:</span> <span class="token number">171</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'QueryPlanningTimeInMillis'<span class="token operator">:</span> <span class="token number">10151</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'ServiceProcessingTimeInMillis'<span class="token operator">:</span> <span class="token number">36</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'WorkGroup'<span class="token operator">:</span> 'primary'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'EngineVersion'<span class="token operator">:</span> <span class="token punctuation">{</span>'SelectedEngineVersion'<span class="token operator">:</span> 'AUTO'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'EffectiveEngineVersion'<span class="token operator">:</span> 'Athena engine version <span class="token number">2</span>'<span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'ResponseMetadata'<span class="token operator">:</span> <span class="token punctuation">{</span>'RequestId'<span class="token operator">:</span> 'foo'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'HTTPStatusCode'<span class="token operator">:</span> <span class="token number">200</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'HTTPHeaders'<span class="token operator">:</span> <span class="token punctuation">{</span>'content-type'<span class="token operator">:</span> 'application/x-amz-json<span class="token number">-1.1</span>'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'date'<span class="token operator">:</span> 'Mon<span class="token punctuation">,</span> <span class="token number">16</span> Aug <span class="token number">2021</span> <span class="token number">23</span><span class="token operator">:</span><span class="token number">42</span><span class="token operator">:</span><span class="token number">11</span> GMT'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'x-amzn-requestid'<span class="token operator">:</span> 'foo'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'content-length'<span class="token operator">:</span> '<span class="token number">2482</span>'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'connection'<span class="token operator">:</span> 'keep-alive'<span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'RetryAttempts'<span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">}</span><span class="token punctuation">}</span></span></code></pre>
<p>Here is an example of a reponse that was <strong>successfull</strong></p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span>'QueryExecution'<span class="token operator">:</span> <span class="token punctuation">{</span>'QueryExecutionId'<span class="token operator">:</span> 'baf0294e-d839-47b6-be58-ca14615e4794'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'Query'<span class="token operator">:</span> <span class="token string">"SELECT complained_email_address FROM complain_count WHERE ingest_timestamp >= NOW() - interval '3' day AND application_id = '123456789' GROUP BY complained_email_address;"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'StatementType'<span class="token operator">:</span> 'DML'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'ResultConfiguration'<span class="token operator">:</span> <span class="token punctuation">{</span>'OutputLocation'<span class="token operator">:</span> 's3<span class="token operator">:</span><span class="token comment">//pinpoint-events-data/application_1/complaints/baf0294e-d839-47b6-be58-ca14615e4794.csv'},</span></span>
<span class="highlight-line"> 'QueryExecutionContext'<span class="token operator">:</span> <span class="token punctuation">{</span>'Database'<span class="token operator">:</span> 'pinpoint_events-data'<span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'Status'<span class="token operator">:</span> <span class="token punctuation">{</span>'State'<span class="token operator">:</span> 'SUCCEEDED'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'SubmissionDateTime'<span class="token operator">:</span> datetime.datetime(<span class="token number">2021</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">,</span> <span class="token number">23</span><span class="token punctuation">,</span> <span class="token number">42</span><span class="token punctuation">,</span> <span class="token number">47</span><span class="token punctuation">,</span> <span class="token number">594000</span><span class="token punctuation">,</span> tzinfo=tzlocal())<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'CompletionDateTime'<span class="token operator">:</span> datetime.datetime(<span class="token number">2021</span><span class="token punctuation">,</span> <span class="token number">8</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">,</span> <span class="token number">23</span><span class="token punctuation">,</span> <span class="token number">42</span><span class="token punctuation">,</span> <span class="token number">50</span><span class="token punctuation">,</span> <span class="token number">720000</span><span class="token punctuation">,</span> tzinfo=tzlocal())<span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'Statistics'<span class="token operator">:</span> <span class="token punctuation">{</span>'EngineExecutionTimeInMillis'<span class="token operator">:</span> <span class="token number">2937</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'DataScannedInBytes'<span class="token operator">:</span> <span class="token number">6202</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'TotalExecutionTimeInMillis'<span class="token operator">:</span> <span class="token number">3126</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'QueryQueueTimeInMillis'<span class="token operator">:</span> <span class="token number">134</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'QueryPlanningTimeInMillis'<span class="token operator">:</span> <span class="token number">1929</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'ServiceProcessingTimeInMillis'<span class="token operator">:</span> <span class="token number">55</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'WorkGroup'<span class="token operator">:</span> 'primary'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'EngineVersion'<span class="token operator">:</span> <span class="token punctuation">{</span>'SelectedEngineVersion'<span class="token operator">:</span> 'AUTO'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'EffectiveEngineVersion'<span class="token operator">:</span> 'Athena engine version <span class="token number">2</span>'<span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'ResponseMetadata'<span class="token operator">:</span> <span class="token punctuation">{</span>'RequestId'<span class="token operator">:</span> 'foo'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'HTTPStatusCode'<span class="token operator">:</span> <span class="token number">200</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'HTTPHeaders'<span class="token operator">:</span> <span class="token punctuation">{</span>'content-type'<span class="token operator">:</span> 'application/x-amz-json<span class="token number">-1.1</span>'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'date'<span class="token operator">:</span> 'Mon<span class="token punctuation">,</span> <span class="token number">16</span> Aug <span class="token number">2021</span> <span class="token number">23</span><span class="token operator">:</span><span class="token number">43</span><span class="token operator">:</span><span class="token number">03</span> GMT'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'x-amzn-requestid'<span class="token operator">:</span> 'foo'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'content-length'<span class="token operator">:</span> '<span class="token number">1895</span>'<span class="token punctuation">,</span></span>
<span class="highlight-line"> 'connection'<span class="token operator">:</span> 'keep-alive'<span class="token punctuation">}</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> 'RetryAttempts'<span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">}</span><span class="token punctuation">}</span></span></code></pre>
<p>You can see above the <code>ResultConfiguration</code> contains the location of the new file with the filename that was automatically generated.</p>
<p>Example:</p>
<pre class="language-text"><code class="language-text"><span class="highlight-line">s3://pinpoint-events-data/application_1/complaints/baf0294e-d839-47b6-be58-ca14615e4794.csv</span></code></pre>
<p>You can now load up your S3 bucket and see the data there in CSV format ready for you. But now that we have data there, we'll need to process it. The way that AWS Lambda's work is by doing a simple process really well, so next up, we need to trigger a lambda to run when a file is added to this S3 location, parse the data and submit it to a new destination.</p>
<h2>Lambda 2: Load the new data from S3, parse it and send to an API endpoint for processing (Python)</h2>
<p>Now we're going to grab the file contents for a specific file in S3 and process it.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">import</span> boto3</span>
<span class="highlight-line"></span>
<span class="highlight-line">s3_client <span class="token operator">=</span> boto3.resource<span class="token punctuation">(</span><span class="token string">'s3'</span><span class="token punctuation">)</span></span>
<span class="highlight-line">bucket <span class="token operator">=</span> <span class="token string">'pinpoint-events-data'</span></span>
<span class="highlight-line">key <span class="token operator">=</span> <span class="token string">'application_1/complaints/baf0294e-d839-47b6-be58-ca14615e4794.csv'</span></span>
<span class="highlight-line">s3_obj <span class="token operator">=</span> s3_client.Object<span class="token punctuation">(</span>bucket, key.replace<span class="token punctuation">(</span><span class="token string">'+'</span>, <span class="token string">' '</span><span class="token punctuation">))</span></span>
<span class="highlight-line">data <span class="token operator">=</span> s3_obj.get<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token string">'Body'</span><span class="token punctuation">]</span>.read<span class="token punctuation">(</span><span class="token punctuation">)</span></span></code></pre>
<p>So far we've connected to S3 and have loaded the contents of the file into the <code>data</code> variable. Now we need to loop through each row of data.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token comment"># helpful package with helpers for handling csv files</span></span>
<span class="highlight-line"><span class="token function">import</span> csv</span>
<span class="highlight-line"></span>
<span class="highlight-line">data_csv <span class="token operator">=</span> csv.DictReader<span class="token punctuation">(</span>data.decode<span class="token punctuation">(</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>.split<span class="token punctuation">(</span><span class="token string">'\n'</span><span class="token punctuation">)</span>, <span class="token assign-left variable">delimiter</span><span class="token operator">=</span><span class="token string">','</span><span class="token punctuation">)</span></span>
<span class="highlight-line">rows <span class="token operator">=</span> <span class="token punctuation">[</span>l <span class="token keyword">for</span> <span class="token for-or-select variable">l</span> <span class="token keyword">in</span> data_csv<span class="token punctuation">]</span></span></code></pre>
<p>Now calling the <code>rows</code> variable, you should see your csv data output something like this.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token punctuation">[</span>OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email1@email.com'</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email2@email.com'</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email3@email.com'</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email4@email.com'</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email5@email.com'</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">]</span></span></code></pre>
<p>Essentially now you have an array you can use which in python is called a <a href="https://www.w3schools.com/python/python_dictionaries.asp">dictionary</a>. So you can call specific results, such as row 2, with the following.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">rows<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span></span></code></pre>
<p>The first dictionary item starts at <code>0</code> just like most other languages. If you had additional columns in your data, for example, <code>user_id</code>...</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token punctuation">[</span>OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email1@email.com'</span><span class="token punctuation">)</span>,<span class="token punctuation">(</span><span class="token string">'user_id'</span>, <span class="token number">187</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span></span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email2@email.com'</span><span class="token punctuation">)</span>,<span class="token punctuation">(</span><span class="token string">'user_id'</span>, <span class="token number">223</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email3@email.com'</span><span class="token punctuation">)</span>,<span class="token punctuation">(</span><span class="token string">'user_id'</span>, <span class="token number">346</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email4@email.com'</span><span class="token punctuation">)</span>,<span class="token punctuation">(</span><span class="token string">'user_id'</span>, <span class="token number">448</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>,</span>
<span class="highlight-line">OrderedDict<span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">'complained_email_address'</span>, <span class="token string">'email5@email.com'</span><span class="token punctuation">)</span>,<span class="token punctuation">(</span><span class="token string">'user_id'</span>, <span class="token number">599</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">]</span></span></code></pre>
<p>You would simply call</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">rows<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token string">'user_id'</span><span class="token punctuation">]</span></span></code></pre>
<p>This would output <code>223</code></p>
<p>(to be continued!)</p>
Setup an Interactive Python Shell on Your Local Machine2021-08-16T00:00:00Zhttps://www.robkjohnson.com/posts/setup-an-interactive-python-shell-on-your-local-machine/<p>Python has gained support and popularity in the development space due to its powerful ability to perform statistical calculations along with enterprise support from the likes of AWS in using it for ETL processes.</p>
<p>Below is an overview of the basics, along with a lambda script overview.</p>
<h2>Bash Terminal</h2>
<p>First open up a <a href="https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/">bash shell if you're on Windows</a>, or the "Terminal" application if you're on Mac OS.</p>
<p>Find out the version you're running by entering:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">python <span class="token parameter variable">--version</span></span></code></pre>
<p>Currently, I'm using <code>Python 2.7.18</code></p>
<p>Anything 2+ is fine for what we're doing. If you need to upgrade, <a href="https://www.python.org/downloads/">download the latest release here</a>.</p>
<h2>Keeping it Simple with iPython</h2>
<p>iPython is an enhanced interactive shell for Python. It adds improved syntax highlighting and tools far beyong what we're doing today, but worth installing to improve the developer experience. Additional benefits include (<a href="https://ipython.org/">from project homepage</a>):</p>
<ul>
<li>A powerful interactive shell.</li>
<li>A kernel for Jupyter.</li>
<li>Support for interactive data visualization and use of GUI toolkits.</li>
<li>Flexible, embeddable interpreters to load into your own projects.</li>
<li>Easy to use, high performance tools for parallel computing.</li>
</ul>
<p>Type the following to initiate the install</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">pip3 <span class="token function">install</span> ipython</span></code></pre>
<p>Verify the location of your install</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">which</span> ipython</span></code></pre>
<p>You should see something like <code>~/local/bin/ipython</code></p>
<p>Now start the shell by simply typing</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">ipython</span></code></pre>
<p>This has now initiated an interactive shell for Python using iPython. You can enter the code you want to use in your script.</p>
<p>Now you could type Python as you would expect. Simple example below (hit enter after each line):</p>
<pre class="language-python"><code class="language-python"><span class="highlight-line">order_1 <span class="token operator">=</span> <span class="token number">120</span></span>
<span class="highlight-line">order_2 <span class="token operator">=</span> <span class="token number">300</span></span>
<span class="highlight-line">total <span class="token operator">=</span> order_1 <span class="token operator">+</span> order_2</span>
<span class="highlight-line"><span class="token comment"># total would now equal 420</span></span>
<span class="highlight-line"><span class="token comment"># show the variable value by typing the variable name and hitting enter</span></span>
<span class="highlight-line">total</span></code></pre>
<p>This is a simple way to get started with Python and learn the basics.</p>
<p>You can also <code>import</code> packages that will help you do more such as <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">boto3</a>, which is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">import</span> boto3</span></code></pre>
<h2>Fun bonus for AWS</h2>
<p>You can follow the same process here for an AWS Cloud Shell which is located at the top right of the AWS Console. This Cloud Shell will inherit all the same permissions that your AWS Console user has.</p>
Using Common Table Expressions in SQL to Improve Your ETL Process2021-08-05T00:00:00Zhttps://www.robkjohnson.com/posts/sql-common-table-expression-overview/<p>At the core of any application, reporting suite, analytics platform or integration service are data set(s). Most often, this data set is stored in a database which can be queried via SQL. Even non-traditional data stores can offer some form of SQL syntax that can be used to query stored data. SQL is a powerful language that anyone who is seeking to integrate, analyze or develop applications should consider being well-versed in its fundamentals.</p>
<h2>Extract, Transform and Load (ETL)</h2>
<p>ETL is an effective approach to integrating your data into other systems and services. The process is as simple as the following:</p>
<ul>
<li><strong>Extract</strong>: get the data</li>
<li><strong>Transform</strong>: shape the data</li>
<li><strong>Load</strong>: load the transformed data to the new location</li>
</ul>
<p>Sounds easy, right? Conceptually it is. The fun is <em>understanding</em> the data sufficiently to know what you should be extracting, how you should be transforming and what assumptions the new system will have when you load the data.</p>
<h2>Using Common Table Expressions (CTE) to simplify your Extract process (and do some Transform too)</h2>
<p>Common table expressions allow you to create temporary tables that can be referenced later in your SQL statement. You write a CTE in the same way you'd write a subquery, except you wrap all your CTE's in a single <code>WITH()</code> clause and define the new temporary table name. I'm using the table name <code>new_cte_table</code> as an example.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">WITH</span> <span class="token punctuation">(</span></span>
<span class="highlight-line"> new_cte_table <span class="token keyword">as</span> <span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token comment">-- type your SELECT query here</span></span>
<span class="highlight-line"> <span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token punctuation">)</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> new_cte_table<span class="token punctuation">.</span>column1</span>
<span class="highlight-line"> <span class="token comment">-- above is how you'd refer to your CTE table</span></span></code></pre>
<h3>Snowflake Syntax for Common Table Expressions</h3>
<p>I noticed when using Snowflake that the <a href="https://docs.snowflake.com/en/user-guide/queries-cte">syntax is a little different</a>. I've included optional column names in the statement too.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">WITH</span></span>
<span class="highlight-line"> new_cte_table <span class="token punctuation">(</span>column_name_1<span class="token punctuation">,</span> column_name_2<span class="token punctuation">,</span> column_name_3<span class="token punctuation">,</span> etc<span class="token punctuation">)</span> <span class="token keyword">as</span> <span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token comment">-- type your SELECT query here</span></span>
<span class="highlight-line"> <span class="token punctuation">)</span></span></code></pre>
<h2>Example Excercise</h2>
<p>Let's say you have been asked to grab all the customers from a database with a <code>customer_class</code> of <code>'vip'</code> who purchased something in 2020. Traditionally you'd write the following <code>SELECT</code> statement.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> c<span class="token punctuation">.</span>customer_id <span class="token keyword">as</span> customer_id</span>
<span class="highlight-line"> <span class="token punctuation">,</span> c<span class="token punctuation">.</span>email <span class="token keyword">as</span> customer_email</span>
<span class="highlight-line"> <span class="token punctuation">,</span> <span class="token function">sum</span><span class="token punctuation">(</span>sh<span class="token punctuation">.</span>total<span class="token punctuation">)</span> <span class="token keyword">as</span> customer_spend</span>
<span class="highlight-line"><span class="token keyword">FROM</span> customers <span class="token keyword">AS</span> c</span>
<span class="highlight-line"><span class="token keyword">JOIN</span> sales_headers <span class="token keyword">AS</span> sh <span class="token keyword">ON</span> sh<span class="token punctuation">.</span>customer_id <span class="token operator">=</span> c<span class="token punctuation">.</span>customer_id</span>
<span class="highlight-line"><span class="token keyword">WHERE</span></span>
<span class="highlight-line"> c<span class="token punctuation">.</span>customer_class <span class="token operator">=</span> <span class="token string">'vip'</span></span>
<span class="highlight-line"> <span class="token operator">AND</span> <span class="token keyword">date</span><span class="token punctuation">(</span>sh<span class="token punctuation">.</span>completed_at<span class="token punctuation">)</span> <span class="token operator">between</span> <span class="token string">'2020-01-01'</span> <span class="token operator">and</span> <span class="token string">'2020-12-31'</span></span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
<p>This works just fine for this specific request.</p>
<blockquote>
<p>Uhm, so why would you want to use CTE's instead of using simple <code>JOIN</code> statements?</p>
</blockquote>
<p>The issue comes when you need to add more complexity. This especially occurs when you need to change the <a href="https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/grain/">grain of data</a>.</p>
<blockquote>
<p>Declaring the grain is the pivotal step in a dimensional design. The grain establishes exactly what a single fact table row represents.</p>
</blockquote>
<p class="caption"><a href="https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/grain/" title="Kimball Group definition of data grain">Kimball Group</a></p>
<p>When you've been asked to add an additional column that requires an aggregate sum of all the rows for a calculation, you're being asked to change the grain. The <code>sales_header</code> grain is order level. The new report that you're being asked to make is aggregating all sales by customer and creating a new measure that shows the percentage of "vip" customer spend in comparison to all customer spend. Ultimately, you're being asked to <em>flatten</em> normalized data. This just means you're taking data from different tables and turning it into one table. This is also known as <em>shaping your data</em>.</p>
<p>Now we know we need three things:</p>
<ol>
<li>One row per customer (only VIP customers)</li>
<li>Sum of total sales per customer (only VIP customers)</li>
<li>Total sales of <em>all</em> customers, which we'll then use to calculate the percentage of sales each VIP customer is contributing individually.</li>
</ol>
<h2>Creating the query</h2>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token comment">-- GRAB ALL OF THE SALES DATA FOR 2020</span></span>
<span class="highlight-line"><span class="token keyword">WITH</span> </span>
<span class="highlight-line">sales_header_data <span class="token keyword">as</span> <span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token keyword">SELECT</span></span>
<span class="highlight-line"> sh<span class="token punctuation">.</span>customer_id <span class="token keyword">as</span> customer_id</span>
<span class="highlight-line"> <span class="token punctuation">,</span> sh<span class="token punctuation">.</span>total <span class="token keyword">as</span> order_total</span>
<span class="highlight-line"> <span class="token punctuation">,</span> sh<span class="token punctuation">.</span>completed_at <span class="token keyword">as</span> order_date</span>
<span class="highlight-line"> <span class="token keyword">FROM</span> sales_header sh</span>
<span class="highlight-line"> <span class="token keyword">WHERE</span></span>
<span class="highlight-line"> <span class="token keyword">date</span><span class="token punctuation">(</span>sh<span class="token punctuation">.</span>completed_at<span class="token punctuation">)</span> <span class="token operator">between</span> <span class="token string">'2020-01-01'</span> <span class="token operator">and</span> <span class="token string">'2020-12-31'</span></span>
<span class="highlight-line"><span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token punctuation">,</span> sales_aggregate <span class="token keyword">as</span> <span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token keyword">SELECT</span></span>
<span class="highlight-line"> <span class="token function">sum</span><span class="token punctuation">(</span>order_total<span class="token punctuation">)</span> <span class="token keyword">as</span> all_orders_total</span>
<span class="highlight-line"> <span class="token comment">-- use the table we just created to ensure the right filter/dates are applied</span></span>
<span class="highlight-line"> <span class="token keyword">FROM</span> sales_header_data</span>
<span class="highlight-line"><span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token comment">-- GET ALL OF THE CUSTOMER DATA WITH THE FILTER OF "customer_class = 'VIP'"</span></span>
<span class="highlight-line"><span class="token punctuation">,</span> customer_data <span class="token keyword">as</span> <span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token keyword">SELECT</span></span>
<span class="highlight-line"> c<span class="token punctuation">.</span>email <span class="token keyword">as</span> customer_email</span>
<span class="highlight-line"> <span class="token punctuation">,</span> c<span class="token punctuation">.</span>customer_id <span class="token keyword">as</span> customer_id</span>
<span class="highlight-line"> <span class="token keyword">FROM</span> customers c</span>
<span class="highlight-line"> <span class="token keyword">WHERE</span></span>
<span class="highlight-line"> c<span class="token punctuation">.</span>customer_class <span class="token operator">=</span> <span class="token string">'vip'</span></span>
<span class="highlight-line"><span class="token punctuation">)</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment">-- NOW THAT YOU HAVE THE DATA YOU WANT, USE THE SET OF DATA IN THE 'FROM'</span></span>
<span class="highlight-line"><span class="token comment">-- AND YOU'LL BE WORKING ONLY WITH THE DATA IN THAT DATA TABLE/SET </span></span>
<span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> customer_data<span class="token punctuation">.</span>customer_id <span class="token keyword">as</span> customer_id</span>
<span class="highlight-line"> <span class="token punctuation">,</span> customer_data<span class="token punctuation">.</span>customer_email <span class="token keyword">as</span> customer_email</span>
<span class="highlight-line"> <span class="token punctuation">,</span> <span class="token function">sum</span><span class="token punctuation">(</span>sales_header_data<span class="token punctuation">.</span>total<span class="token punctuation">)</span> <span class="token keyword">as</span> customer_spend</span>
<span class="highlight-line"> <span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token function">sum</span><span class="token punctuation">(</span>sales_header_data<span class="token punctuation">.</span>total<span class="token punctuation">)</span> <span class="token operator">/</span> sales_aggregate<span class="token punctuation">.</span>all_orders_total<span class="token punctuation">)</span> <span class="token keyword">as</span> perc_total_customer_spend</span>
<span class="highlight-line"><span class="token keyword">FROM</span> sales_header_data</span>
<span class="highlight-line"><span class="token keyword">JOIN</span> customer_data <span class="token keyword">ON</span> customer_data<span class="token punctuation">.</span>customer_id <span class="token operator">=</span> sales_header_data<span class="token punctuation">.</span>customer_id</span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"> customer_data<span class="token punctuation">.</span>customer_id</span>
<span class="highlight-line"> <span class="token punctuation">,</span> customer_data<span class="token punctuation">.</span>customer_email</span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"><span class="token function">sum</span><span class="token punctuation">(</span>sales_header_data<span class="token punctuation">.</span>total<span class="token punctuation">)</span> <span class="token keyword">DESC</span></span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
<h2>Conclusion</h2>
<p>Think of a common table expression as part of your extraction along with any simple transforms that are needed, such as casting or applying calculations. It's also a way to ensure you simplify the data set that you're extracting. You only need to filter your data once and then you can refer to that data set for aggregates and JOINS later. This is done by using CTE's to establish the base data sets which you can extrapolate from later. Then use your main query to help finish off the shape of your data.</p>
<p>I hope this was helpful. It's helped improved my thinking around how I write SQL statements for extraction and transformation.</p>
How to Use Rollbar for Logging in a Custom App2020-12-27T00:00:00Zhttps://www.robkjohnson.com/posts/logging-with-rollbar/<p>I run an automation service and each night it runs through several cron jobs that ETL data and deliver it to various services. For a long time I used <a href="https://pushover.net/">Pushover</a> to tell me of the status of how many records each job parsed and sent over and alert me if there are any issues.</p>
<h2>Pushover FTW!</h2>
<p>What I like about Pushover is that you can create groups and have alerts and notifications post to a group. It's a very low cost way to have small groups be aware of any problems. In my case, I have it alert product managers, customer service and email marketing campaign managers, so that if a service goes down that impacts customers, they know to delay a campaign, or help navigate an issue with a customer.</p>
<h2>Why Rollbar?</h2>
<p>After the automation service started to increase in its number of jobs, the Pushover notifications coming to my phone started to get noisy. I had enough information to know it was running well, and I still get alerts when issues occur. However, now I wanted to log each job, how many records it parsed, when and what files were created and where it dropped them off.</p>
<p>I started to create a database schema to track these "Sync Events." However, as I was thinking ahead, I thought of the need to create visualizations, error reporting, querying the logs and undoubtably more.</p>
<p>I had played around with Rollbar enough. The pricing is very fair, especially if you're not logging a large amount of data. Rollbar is also very friendly to integrate with and use. It's helped debug certain issues quickly as you can verify what occurred and at what time. It's been a good upgrade.</p>
<h2>Integrating Rollbar with PHP</h2>
<p>I followed the basic PHP installation and setup in the <a href="https://docs.rollbar.com/docs/basic-php-installation-setup">Rollbar docs</a>.</p>
<p>I installed Rollbar using <a href="https://getcomposer.org/">Composer</a>, which automatically handles loading the classes into the project.</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"require"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token property">"rollbar/rollbar"</span><span class="token operator">:</span> <span class="token string">"^1"</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The I added the Rollbar config to my own app configuration file that is loaded on each page and job that runs on the app.</p>
<pre class="language-php"><code class="language-php"><span class="highlight-line"><span class="token keyword">use</span> <span class="token package"><span class="token punctuation">\</span>Rollbar<span class="token punctuation">\</span>Rollbar</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token variable">$config</span> <span class="token operator">=</span> <span class="token keyword">array</span><span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token comment">// required</span></span>
<span class="highlight-line"> <span class="token string single-quoted-string">'access_token'</span> <span class="token operator">=></span> <span class="token string single-quoted-string">'YOUR_ACCESS_TOKEN'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token comment">// optional - environment name. any string will do.</span></span>
<span class="highlight-line"> <span class="token string single-quoted-string">'environment'</span> <span class="token operator">=></span> <span class="token string single-quoted-string">'YOUR_ENVIRONMENT_NAME'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token comment">// optional - path to directory your code is in. used for linking stack traces.</span></span>
<span class="highlight-line"> <span class="token string single-quoted-string">'root'</span> <span class="token operator">=></span> <span class="token string single-quoted-string">'/var/www/myapp'</span></span>
<span class="highlight-line"><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token class-name static-context">Rollbar</span><span class="token operator">::</span><span class="token function">init</span><span class="token punctuation">(</span><span class="token variable">$config</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>This took care of the basics of application errors, warnings and critical events. But what I wanted to do was send specific informational events with additional data. Rollbar is fantastic at handling this, as you can send an array over in your message/log and it will automatically save the data and allow you to view and query that data in Rollbar. This can also allow you to setup specific notifications based on certain events. This is what got me most excited about using Rollbar.</p>
<h2>Sending a custom message to Rollbar</h2>
<p>Below you can see that I post a message at a level of <code>INFO</code> and then send over an array of data which can then be stored as an object along with the event.</p>
<pre class="language-php"><code class="language-php"><span class="highlight-line"><span class="token comment">// LOG TO ROLLBAR</span></span>
<span class="highlight-line"><span class="token class-name static-context">Rollbar</span><span class="token operator">::</span><span class="token function">log</span><span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token class-name static-context">Level</span><span class="token operator">::</span><span class="token constant">INFO</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"Cron: <span class="token interpolation"><span class="token variable">$service</span></span> <span class="token interpolation"><span class="token variable">$sync_event_name</span></span> (<span class="token interpolation"><span class="token variable">$status</span></span>)"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token comment">// key-value additional data</span></span>
<span class="highlight-line"> <span class="token keyword">array</span><span class="token punctuation">(</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"sync_event"</span> <span class="token operator">=></span> <span class="token string double-quoted-string">"<span class="token interpolation"><span class="token variable">$sync_event_name</span></span>"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"service"</span> <span class="token operator">=></span> <span class="token variable">$service</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"service_location"</span> <span class="token operator">=></span> <span class="token variable">$_SERVER</span><span class="token punctuation">[</span><span class="token string single-quoted-string">'PHP_SELF'</span><span class="token punctuation">]</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"records_sent"</span> <span class="token operator">=></span> <span class="token variable">$records_sent</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"success"</span> <span class="token operator">=></span> <span class="token variable">$status</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"duration_seconds"</span> <span class="token operator">=></span> <span class="token variable">$duration_seconds</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string double-quoted-string">"file_name"</span> <span class="token operator">=></span> <span class="token variable">$csv_filename</span></span>
<span class="highlight-line"> <span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<h2>How Rollbar categorizes things: Items vs Occurances</h2>
<p>Each message/log that is sent over from your app to Rollbar will be classified and grouped as an "Item." This means if you send over two logs that have the same message, it will categorize them as one item. This helps Rollbar determine how frequently a particular item is occuring.</p>
<p>Occurances are unique instances of log events that can roll up into an item. This helps you look into each specific occurance.</p>
<p>For example, if you go into an item in Rollbar, you will see a tab called "Occurances." Here you will see a list of each occurance of an item. It has some basic information already for each item, such as a Timestamp of the occurance, the IP of the host etc.</p>
<p>Along with each occurance you will also see additional fields under <code>extra</code>. Each array key and value can be access under the <code>extra</code> object in Rollbar. For example, above we are sending over <code>records_sent</code>. We can see this data in the <code>extra.records_sent</code> column and it is now also queryable in RQL (Rollbar Query Language)</p>
<h2>Rollbar RQL (Rollbar Query Language)</h2>
<p>Rollbar has its own SQL interface for querying log data called RQL. It's effectively vanilla SQL that has syntax very similar to MySQL or PostGres.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span> <span class="token operator">*</span></span>
<span class="highlight-line"><span class="token keyword">FROM</span> item_occurrence</span>
<span class="highlight-line"><span class="token keyword">WHERE</span> item<span class="token punctuation">.</span>counter <span class="token operator">=</span> <span class="token number">88</span></span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span> <span class="token keyword">timestamp</span> <span class="token keyword">DESC</span></span>
<span class="highlight-line"><span class="token keyword">LIMIT</span> <span class="token number">20</span></span></code></pre>
<p>In the above SQL statement we can pull all occurences of item number 88. (Rollbar automatically numbers items/messages it receives and does its best to group them together.) If you have a good naming convention for your messages, it greatly helps Rollbar in how it groups them.</p>
<p>If we want to see the number of occurances for a specific item in the last 24 hours we could simply change the <code>WHERE</code> statement to be:</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span> <span class="token operator">*</span></span>
<span class="highlight-line"><span class="token keyword">FROM</span> item_occurrence</span>
<span class="highlight-line"><span class="token keyword">WHERE</span> item<span class="token punctuation">.</span>counter <span class="token operator">=</span> <span class="token number">88</span></span>
<span class="highlight-line"> <span class="token operator">AND</span> <span class="token keyword">timestamp</span> <span class="token operator">></span> unix_timestamp<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">24</span></span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span> <span class="token keyword">timestamp</span> <span class="token keyword">DESC</span></span>
<span class="highlight-line"><span class="token keyword">LIMIT</span> <span class="token number">20</span></span></code></pre>
<h2>Conclusion</h2>
<p>Rollbar is a very cost effective solution for logging management. Like any logging service, taking care to set it up well can help many teams understand what is happening and reducing needless messages/items will help ensure that it won't get to a noise level where people ignore all logs, even the important ones!</p>
<p>Setting up custom RQL statements and notifications can help automate your application alerts so that you can get notified of specific issues, like credit card fraud, or third-party service endpoints failing.</p>
Creating an SQL query where one column contains aggregated data from another table2020-12-15T00:00:00Zhttps://www.robkjohnson.com/posts/comma-separated-string-from-subquery-sql/<p>Every now and then there can be a need to create a comma separated string that aggregates information from another table. Below is an example scenario.</p>
<h2>Work request</h2>
<p>We have been asked to list all the pick zones for the items in cartons that are <code>in_distribution</code>. Often a carton contains items that are in more than one pick zone.</p>
<p>The following columns are being requested.</p>
<ol>
<li><code>cartons.carton_id</code> the number of the carton/box that the items will be contained in</li>
<li><code>cartons.status</code> the status of the carton <code>['pending', 'in_distribution', 'fulfilled']</code></li>
<li><code>zones.code</code> in a comma separated string showing each zone where an item is located for the carton</li>
<li><code>wave.id</code> the ID of the wave which represents the batch of item picks given to the warehouse floor to be fulfilled</li>
<li>Number of items within the carton which will need to be summed from <code>carton_items.id</code> which has a unique ID for each item, regardless of <code>SKU</code> or <code>stock_item.id</code></li>
</ol>
<h2>Database Schema</h2>
<p>We have the following tables:</p>
<ol>
<li><code>cartons</code> holding information for a carton/box that holds items</li>
<li><code>carton_items</code> holding what items are in what cartons/box</li>
<li><code>waves</code> holding information on distribution batches that are given to the warehouse floor</li>
<li><code>zones</code> holding information about what zones items are located in within a location/warehouse</li>
<li><code>locations</code> holding information for each warehouse holding stock</li>
</ol>
<p>In a location/warehouse you often have the same stock item located on different shelves within the same warehouse. It may sound counterintuitive, but having multiple locations for popular stock items can greatly speed up the fulfillment process. Usually high demand stock items are paired closely together in order to increase pick efficiency and ultimately faster fulfillment times.</p>
<h2>The problem</h2>
<p>If a carton contains items that have more than one zone, they will duplicate rows. We've been asked specifically to list all the zones in the same column and only have one row per carton.</p>
<h2>Incorrect query</h2>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span>id<span class="token punctuation">,</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span><span class="token keyword">status</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> zones<span class="token punctuation">.</span>code<span class="token punctuation">,</span></span>
<span class="highlight-line"> waves<span class="token punctuation">.</span>id<span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token function">sum</span><span class="token punctuation">(</span>carton_items<span class="token punctuation">.</span>quantity<span class="token punctuation">)</span> <span class="token keyword">as</span> item_quantity</span>
<span class="highlight-line"><span class="token keyword">FROM</span> cartons <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token keyword">JOIN</span> waves <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span> <span class="token keyword">ON</span> waves<span class="token punctuation">.</span>id <span class="token operator">=</span> cartons<span class="token punctuation">.</span>wave_id</span>
<span class="highlight-line"><span class="token keyword">JOIN</span> carton_items <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span> <span class="token keyword">ON</span> carton_items<span class="token punctuation">.</span>carton_id <span class="token operator">=</span> cartons<span class="token punctuation">.</span>id</span>
<span class="highlight-line"><span class="token keyword">JOIN</span> zones <span class="token keyword">WITH</span> <span class="token punctuation">(</span><span class="token keyword">NO</span> <span class="token keyword">LOCK</span><span class="token punctuation">)</span> <span class="token keyword">ON</span> zones<span class="token punctuation">.</span>id <span class="token operator">=</span> carton_items<span class="token punctuation">.</span>zone_id</span>
<span class="highlight-line"><span class="token keyword">WHERE</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span><span class="token keyword">status</span> <span class="token operator">=</span> <span class="token string">'in_distribution'</span></span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span>id<span class="token punctuation">,</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span><span class="token keyword">status</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> zones<span class="token punctuation">.</span>code<span class="token punctuation">,</span></span>
<span class="highlight-line"> waves<span class="token punctuation">.</span>id</span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"> waves<span class="token punctuation">.</span>id <span class="token keyword">ASC</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span>id <span class="token keyword">ASC</span></span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
<h2>Incorrect query output</h2>
<p>The below output shows that we would get 3 rows back from this query which is not the desired result.</p>
<pre class="language-text"><code class="language-text"><span class="highlight-line">|------------|---------------|-----------------|----------|--------------|</span>
<span class="highlight-line">|cartons.id |cartons.status |zones.code |waves.id |item_quantity |</span>
<span class="highlight-line">|------------|---------------|-----------------|----------|--------------|</span>
<span class="highlight-line">|12345 |in_distribution|OR |987654321 |12 |</span>
<span class="highlight-line">|12345 |in_distribution|GR |987654321 |12 |</span>
<span class="highlight-line">|12345 |in_distribution|BL |987654321 |12 |</span></code></pre>
<p>Instead we want to combine all the results in <code>zones.code</code> so that only one carton appears per row.</p>
<h2>Solution query</h2>
<p>Because we're going to combine all <code>zones.code</code> into one string, I'm going to change the name of the column to <code>zones_list</code>. We're going to acheive this by creating a subquery that grabs all of the <code>zones</code> connected to <code>carton_items</code>. We'll be using two SQL functions <code>STUFF()</code> and <code>FOR XML PATH</code> to ensure the string is properly comma delimited.</p>
<pre class="language-sql"><code class="language-sql"><span class="highlight-line"><span class="token keyword">SELECT</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span>id<span class="token punctuation">,</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span><span class="token keyword">status</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> STUFF<span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token keyword">SELECT</span> <span class="token string">', '</span> <span class="token operator">+</span> z<span class="token punctuation">.</span>code</span>
<span class="highlight-line"> <span class="token keyword">FROM</span> cartons <span class="token keyword">AS</span> c <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token keyword">JOIN</span> carton_items <span class="token keyword">AS</span> ci <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span> <span class="token keyword">ON</span> ci<span class="token punctuation">.</span>carton_id <span class="token operator">=</span> c<span class="token punctuation">.</span>id</span>
<span class="highlight-line"> <span class="token keyword">JOIN</span> zones <span class="token keyword">AS</span> z <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span> <span class="token keyword">ON</span> ci<span class="token punctuation">.</span>zone_id <span class="token operator">=</span> z<span class="token punctuation">.</span>id </span>
<span class="highlight-line"> <span class="token keyword">WHERE</span> c<span class="token punctuation">.</span>id <span class="token operator">=</span> cartons<span class="token punctuation">.</span>id</span>
<span class="highlight-line"> <span class="token keyword">GROUP</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"> z<span class="token punctuation">.</span>code</span>
<span class="highlight-line"> <span class="token keyword">FOR</span> XML PATH<span class="token punctuation">(</span><span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span> <span class="token keyword">AS</span> zones_list<span class="token punctuation">,</span></span>
<span class="highlight-line"> waves<span class="token punctuation">.</span>id<span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token function">sum</span><span class="token punctuation">(</span>carton_items<span class="token punctuation">.</span>quantity<span class="token punctuation">)</span> <span class="token keyword">as</span> item_quantity</span>
<span class="highlight-line"><span class="token keyword">FROM</span> cartons <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span></span>
<span class="highlight-line"><span class="token keyword">JOIN</span> waves <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span> <span class="token keyword">ON</span> waves<span class="token punctuation">.</span>id <span class="token operator">=</span> cartons<span class="token punctuation">.</span>wave_id</span>
<span class="highlight-line"><span class="token keyword">JOIN</span> carton_items <span class="token keyword">WITH</span> <span class="token punctuation">(</span>NOLOCK<span class="token punctuation">)</span> <span class="token keyword">ON</span> carton_items<span class="token punctuation">.</span>carton_id <span class="token operator">=</span> cartons<span class="token punctuation">.</span>id</span>
<span class="highlight-line"><span class="token keyword">WHERE</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span><span class="token keyword">status</span> <span class="token operator">=</span> <span class="token string">'in_distribution'</span></span>
<span class="highlight-line"><span class="token keyword">GROUP</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span>id<span class="token punctuation">,</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span><span class="token keyword">status</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> waves<span class="token punctuation">.</span>id</span>
<span class="highlight-line"><span class="token keyword">ORDER</span> <span class="token keyword">BY</span></span>
<span class="highlight-line"> waves<span class="token punctuation">.</span>id <span class="token keyword">ASC</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> cartons<span class="token punctuation">.</span>id <span class="token keyword">ASC</span></span>
<span class="highlight-line"><span class="token punctuation">;</span></span></code></pre>
<h2>Solution query output</h2>
<p>If there are three zones connected to a carton OR, GR, BL we now get a single row returned with a comma separated list in the form of a string as requested.</p>
<pre class="language-text"><code class="language-text"><span class="highlight-line">|------------|---------------|-----------------|----------|--------------|</span>
<span class="highlight-line">|cartons.id |cartons.status |zones_list |waves.id |item_quantity |</span>
<span class="highlight-line">|------------|---------------|-----------------|----------|--------------|</span>
<span class="highlight-line">|12345 |in_distribution|OR, GR, BL |987654321 |12 |</span></code></pre>
<h2>Explaining the syntax</h2>
<p>The <code>STUFF() function</code> allows us to create a string by "stuffing" things inside of it.</p>
<p>There are 3 parameters which are:</p>
<ol>
<li>the string we want to stuff (in our case it's the string we're creating from the subquery)</li>
<li>the location we want to start deleting and inserting characters (in our case the first character)</li>
<li>the number of characters we want to delete (the first character being output by our subquery is a <code>,</code> and we definitely want to delete that)</li>
</ol>
<p><code>FOR XML PATH('')</code> at the end of the query is a function used to create XML from a query. By declaring an empty string in the function <code>''</code> we can comma delimit each entry.</p>
<p>The importance of the <code>STUFF()</code> function is that it removes the first comma from the list.</p>
Solidus & Subscriptions: Adventures of a Publisher Trying to Disrupt Itself2020-09-25T00:00:00Zhttps://www.robkjohnson.com/posts/solidus-conf-2020/<p>This year was the first Solidus Conf that was 100% digital, thanks to the worldwide COVID-19 pandemic. You can see tweets on Twitter via the hashtag <a href="https://twitter.com/hashtag/SolidusConf2020" title="solidus conf 2020 tweets">#SolidusConf2020</a></p>
<p><img src="https://www.robkjohnson.com/assets/img/brands/solidus.svg#center" alt="Solidus" /></p>
<p>The Solidus community is fairly small, although the project currently has over <a href="https://github.com/solidusio/solidus">3.3k stars on Github</a>. It's one of the leading open source Ecommerce platforms for the Ruby on Rails stack, the other being the original Ecommerce project, <a href="https://spreecommerce.org/">Spree</a>.</p>
<h2>Spree vs Solidus</h2>
<p>Spree was the original open source Ecommerce project, but it was purchased by another company who worked toward monetizing service integrations through a for-profit, optional service, called Wombat, which took the direction of Spree toward a more inherently commercial direction. This toubled a number of developers and open source advocates, who depended on Spree for many of their clients.</p>
<p>In response to this directional change, a group of agencies and retailers who relied on Spree decided to fork Spree 2.4 and combine resources to ensure the platforms future development efforts were adhering to a less commercially oriented roadmap. This was the beginning of <a href="https://solidus.io/">Solidus</a>.</p>
<p>You can <a href="https://nebulab.it/blog/why-solidus/">read more context here from Nebulab</a>, an agency based in Italy who has been using Spree/Solidus since its inception and is currently the main sponsor of the Solidus open source project.</p>
<h2>My Solidus 2020 Presentation</h2>
<p>I flew to New York in 2014 to scope out Spree and saw a lot of energy behind the platform and our team decided to take the plunge in replatforming. It wasn't an easy transition for many reasons. However, we're in full swing with Solidus now and it has been a reliable platform that's enabled us to lean into the disruptive transformation that continues to force change within the book industry. This is the essence of my presentation that overviews our journey on Solidus while juxtaposing the disruption for a book publisher in Salt Lake City.</p>
<p><a href="https://www.robkjohnson.com/assets/presentations/solidus-2020" class="button download" id="Solidus 2020 Presentation" title="Solidus & Subscriptions: Adventures of a Publisher Trying to Disrupt Itself"><span class="icon is-danger"><strong>▶</strong></span>View the deck</a></p>
<p>As always, feel free to reach out if you have any thoughts or questions.</p>
<h2>Update 2020-10-16: Video Now Available on YouTube</h2>
<p>The Solidus Team now has the video uploaded to YouTube. Many thanks to <a href="https://twitter.com/Seanphden">@Seanphden</a> for doing such a great job in organizing the event.</p>
<p><a href="https://www.youtube.com/watch?v=p1jnBbJgFGI" class="button download" id="Solidus 2020 Presentation YouTube" title="YouTube video of Solidus & Subscriptions: Adventures of a Publisher Trying to Disrupt Itself" target="_blank"><span class="icon is-danger"><strong>▶</strong></span>Watch on YouTube</a></p>
<p><a href="https://www.robkjohnson.com/assets/presentations/solidus-2020"><img src="https://www.robkjohnson.com/assets/img/content/solidus-subscriptions.png" alt="Solidus Subscriptions" /></a></p>
How to create an email newsletter sign up service Pt. 12020-08-28T00:00:00Zhttps://www.robkjohnson.com/posts/creating-a-newsletter-signup-service/<div class="callout"><span class="jam jam-alert is-info"></span> I wrote this code for a project in 2015. There is likely a much better way today. Still, the principles here remain relevant</div>
<p>Here is a brief overview of how you can create a newsletter form which will post the data entered to an endpoint using javascript. This will enable you to create your own email newsletter subscription service.</p>
<h2>Create a simple HTML form with a message box</h2>
<p>The first thing we want to do here is create an entry form for an email. Unlike usual HTML forms, we won't be using the <code>action</code> method to send data, instead we will create a javascript listener that will pass along what is entered and send it to an endpoint for processing. Here I'm replacing the action with <code>javascript:;</code> which doesn't actually do anything and instead I'm using the <code>onClick="sendSub()"</code> attribute to initiate the javascript function which we'll get to next.</p>
<p>Note that there is a simple <code><div></code> with <code>id="message"</code> below the form which will be used to show a message once we hear back from the endpoint. There is also a hidden input called <code>campaign_id</code> which has a value of <code>1</code>. The purpose of this is so that you can send a newsletter ID along with an email address, which allows you to create different email newsletter subscriptions easily by assigning a different number value.</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>form</span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>subscription<span class="token punctuation">"</span></span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>subscription<span class="token punctuation">"</span></span> <span class="token attr-name">action</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> Email: <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>text<span class="token punctuation">"</span></span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>email<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>br</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>hidden<span class="token punctuation">"</span></span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>campaign_id<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>1<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>input</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>submit<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Send<span class="token punctuation">"</span></span> <span class="token special-attr"><span class="token attr-name">onclick</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token value javascript language-javascript"><span class="token function">sendSub</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span><span class="token punctuation">"</span></span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>form</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>div</span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>message<span class="token punctuation">"</span></span> <span class="token special-attr"><span class="token attr-name">style</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token value css language-css"><span class="token property">color</span><span class="token punctuation">:</span>green<span class="token punctuation">;</span></span><span class="token punctuation">"</span></span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>div</span><span class="token punctuation">></span></span></span></code></pre>
<h2>Write the javascript function</h2>
<p>Now we have the form ready to go, it's now time to write the function that submits the entered values above to an endpoint. The first thing we're doing here is including jQuery, which simplified the ajax call script for us. Ajax stands for "Asynchronous Javascript and XML." Essentially it allows you to run javascript without reloading a page, which is very common in todays applications.</p>
<p>Then we will create a <code>function</code> called <code>sendSub()</code>. We'll now create two variables which will inherit the values we assigned in the above form by using the <code>getElementById().value</code> method. Essentially, we're asking javascript to grab the elements input value from the form and store it in variables so that we can pass it along to the ajax call.</p>
<p>Every endpoint call you make is either <code>POST</code> or <code>GET</code>. <code>POST</code> calls pass along the data inside of the URL. You can often see these types of URL's with parameters such as <code>utm_campaign="christmas_shopping"</code>. The first part of this parameter is a key <code>utm_campaign</code> which you can think of as a variable. Essentially a mini container that can store a value which you can then recall to use somewhere else. <code>christmas_shopping</code> is the value. You'll often hear people refer to <em>"key value pairs"</em> which just means that you have data which can be accessed by calling the key, or variable name, to access the value stored in it.</p>
<p>Now that we've got the variables loaded with the values from the form and we've defined that this will be a <code>POST</code> request, we can now form the URL that will send the data.</p>
<p>Lastly, we will return a message if the endpoint/URL responds with a <code>200</code> code, which means that everything went well.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// include https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js</span></span>
<span class="highlight-line"><span class="token keyword">function</span> <span class="token function">sendSub</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">var</span> email <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">'email'</span><span class="token punctuation">)</span><span class="token punctuation">.</span>value<span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token keyword">var</span> campaign_id <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">'campaign_id'</span><span class="token punctuation">)</span><span class="token punctuation">.</span>value<span class="token punctuation">;</span></span>
<span class="highlight-line"> $<span class="token punctuation">.</span><span class="token function">ajax</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">"POST"</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">url</span><span class="token operator">:</span> <span class="token string">'https://yourdomain.com/?email='</span> <span class="token operator">+</span> email <span class="token operator">+</span> <span class="token string">'&campaign_id='</span> <span class="token operator">+</span> campaign_id<span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token literal-property property">cache</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token function-variable function">success</span><span class="token operator">:</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token function">$</span><span class="token punctuation">(</span><span class="token string">'#message'</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">html</span><span class="token punctuation">(</span><span class="token string">"Success!"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<h2>Wrapping up</h2>
<p>What's great about javascript is that you don't need a server to test this out. You can go ahead and create a local file using a <a href="https://www.techradar.com/best/best-text-editors" title="best text editors for 2020">text editor</a> and run this on your browser for testing. However, we still need to make the endpoint that will receive the data, store it and then respond with a success or error code and message</p>
How to Ensure UTF8 Characters with fputcsv in PHP2019-11-25T00:00:00Zhttps://www.robkjohnson.com/posts/how-to-ensure-utf8-characters-with-fputcsv-in-php/<p>I was working on a project where I transform data, create and load a CSV for another system to process. The issue was non-conforming characters were breaking the endpoint system. I ensured that my php.ini configuration had the UTF8 character set defined as default, however the output from fputcsv() was still outputting non-UTF8 characters.</p>
<p>I found the a <a href="https://stackoverflow.com/questions/21988581/write-utf-8-characters-to-file-with-fputcsv-in-php">Stack Overflow conversation</a> and saw that the fix was to issue the following definitions in the fputcsv() function.</p>
<pre class="language-php"><code class="language-php"><span class="highlight-line"><span class="token function">fprintf</span><span class="token punctuation">(</span><span class="token variable">$df</span><span class="token punctuation">,</span> <span class="token function">chr</span><span class="token punctuation">(</span><span class="token number">0xEF</span><span class="token punctuation">)</span><span class="token operator">.</span><span class="token function">chr</span><span class="token punctuation">(</span><span class="token number">0xBB</span><span class="token punctuation">)</span><span class="token operator">.</span><span class="token function">chr</span><span class="token punctuation">(</span><span class="token number">0xBF</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>The line <code>fprintf($df, chr(0xEF).chr(0xBB).chr(0xBF));</code> writes file header for correct encoding.</p>
<p>Thanks goes to <a href="https://stackoverflow.com/users/2611927/hardy">Hardy</a>.</p>
Ok.com - Creating a Media Ratings & Reviews Platform with User-Generated Content2018-12-05T00:00:00Zhttps://www.robkjohnson.com/posts/okcom-your-family-media-guide/<p>In 2011, I was working at <a href="https://www.deseretdigital.com/">DDM</a> as a Product Manager and was asked to rethink how movie reviews could be improved for the <a href="https://www.deseretnews.com/">Deseret News</a>. With restructuring at the newspaper, there had been changes as the industry sought, and still does, a more viable business model.</p>
<p>One of the most expensive costs for a newspaper is paying for writers, editors, journalists and in this case movie critics. Back in 2012, as Facebook was being greatly admired in the digital space, the concept of user-generated content became a potential answer for solving the cost of fresh and relevant content.</p>
<p>This approach was naturally appealing to the company, as we could create interactive applications that would be more helpful for our audiences which would in-turn increase usage, engagement and product viability.</p>
<p>With the ability to foster two-way communication and help connect communities, empowerment became the key goal.</p>
<div class="callout"><span class="is-info">Fun fact:</span> Utah has the most movie theatres per capita than any other state.</div>
<h2>Thinking about pain</h2>
<p>A Product Manager is always thinking about customer pain and how it can be reduced with a viable solution.</p>
<p>We found two main pain points:</p>
<ol>
<li>Wasting your time and money watching a show you don't like</li>
<li>Watching specific content that makes you feel uncomfortable, especially when watching with your kids</li>
</ol>
<p>You can distill these two pain points into the following customer statement.</p>
<blockquote>
<p>I took my kids to the theatre to watch the new (x) movie, it was $10 a ticket, with another $20 for popcorn and drinks. The show was terrible and I didn't realize how inappropriate some of the scenes were going to be.</p>
</blockquote>
<p class="caption">—Costco Carl</p>
<p>A family of 5 visiting the theatre can cost upwards of $50-$70. A pretty hefty waste of money if you don't know what you're in for. Also, the time can be just as costly for some.</p>
<h2>A family perspective</h2>
<p>Critics are experienced, artistically attuned and very knowledgable about movies. However, we found that they often didn't focus enough on families. Here was an opportunity to provide an application that would allow parents to connect with other parents and like-minded people to help them navigate their movie choices.</p>
<p>
</p><div style="padding:56.25% 0 0 0;position:relative;"><iframe src="https://player.vimeo.com/video/67771064?byline=0&portrait=0" style="position:absolute;top:0;left:0;width:100%;height:100%;" frameborder="0" allow="autoplay; fullscreen" allowfullscreen=""></iframe></div><script src="https://player.vimeo.com/api/player.js"></script>
<p></p>
<p class="caption">Ok.com 30 second spot ad</p>
<h2>Designing a flexible rating system</h2>
<p>Initially, the objective was to consider movies only, but we soon thought about how a rating system could be applied across various media formats. The three biggest formats we thought of were movies, video games and books.</p>
<p>We went to work to figure out a rating system that was intuitive and could overlay on top of currently existing rating systems. So someone could, from a glance, understand a lot about the movie.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/okcom/okcom-rating-widget.png" alt="Ok.com Rating Widget" /></p>
<p class="caption">Ok.com rating widget</p>
<h2>The beginning of Ok.com</h2>
<p>After the idea was presented, and the initial concept was live on <a href="https://www.deseretnews.com/">DeseretNews.com</a>, a generous donor who liked the idea decided to gift the domain <a href="https://ok.com/">Ok.com</a>. It was a significant and surprising contribution that was a great boost for everyone involved.</p>
<p>After a few rounds of design, we implemented the new branding.</p>
<p><img src="https://www.robkjohnson.com/assets/img/brands/ok.svg#small" alt="Ok.com" /></p>
<p><img src="https://www.robkjohnson.com/assets/img/content/okcom/okcom-home.png" alt="Ok.com Homepage" /></p>
<p class="caption">Ok.com home page</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/okcom/eoa-banner-ok.png" alt="Ok.com Ad" /></p>
<p class="caption">Ok.com banner ad</p>
<p><a href="https://www.ksl.com/article/22998224/okcom-movies-rated-by-parents-for-parents" class="download button" title="Watch the KSL News Story"><span class="icon is-danger"><strong>▶</strong></span>Watch the KSL News Story</a></p>
<h2>Features</h2>
<p>A list of the main features we developed for the platform.</p>
<ul>
<li>Nationwide movie times</li>
<li><a href="https://www.redbox.com/">Redbox</a> API for availability</li>
<li><a href="https://www.netflix.com/">Netflix</a> API for availability</li>
<li>Movie recommendation feeds</li>
<li>Facebook Connect with friend list integration</li>
<li>Integrate reviews from other critic sources</li>
<li>Ability to follow anyone, similar to <a href="https://www.twitter.com/">Twitter</a> functionality</li>
<li>Ability to filter reviews and ratings from your friends</li>
<li>Notifications for when a movie comes out at Redbox or Netflix etc</li>
<li>Discussion pages both public and within your friends network</li>
</ul>
<div class="callout"><span class="icon is-danger"><strong>⍟</strong></span> The entire platform was built and run between <a href="https://www.linkedin.com/in/kenneth-ahlstrom-8284511a" title="Ken Ahlstrom">Ken Ahlstrom</a> and myself. Ken did the heavy lifting of the backend development, an impressive feat considering all of the features we wanted!</div>
<h2>Applying several revenue streams</h2>
<p>In order to ensure viability of the platform we worked on the following revenue streams.</p>
<ul>
<li>B2B Licensing: We successfully white-labelled the platform to several media outlets across the country</li>
<li>Display ads</li>
<li>Pre-Roll</li>
<li>Sponsorships</li>
<li>Affiliate revenue</li>
</ul>
<p><img src="https://www.robkjohnson.com/assets/img/content/okcom/GameofShadows.jpg" alt="Game of Shadows" /></p>
<p class="caption">Example affiliate ad</p>
<h2>Learnings</h2>
<p>Ultimately "Ok.com - Your Family's Media Guide" ended up being sunsetted. Although this occurred after my departure from DDM, I believe our business model was too influenced from our news media roots. Depending on pageviews and white-labelling for others who are dependent on pageviews wasn't the best recipe for success.</p>
<p>My recommendation was that we focus on the Ok brand and stop focus on the white-labelling (which incurred additional licensing fees with our 3rd party agreements), increase gamification with quizes and focus on positive user interactive experiences such as sharable social challenges ("What super hero are you?"). Video was also an area we should have focused more on, with pre-roll having the ability to provide a another source of revenue.</p>
<p>Starting a brand from scratch with a business model that is dependent on high volumes of traffic with no substantial funding to drive traffic is a fundamental problem. In the beginning of the web, the saying "if you build it, they will come" was true. In today's over-populated digital world, that is certainly no longer the case. It was from this project that I learned the importance of leveraging audiences and platforms that have already been established.</p>
<p>If you have a current audience, brand or customer list, use them. Use them all in a way that can expand your ecosystem of brands. You'll reap faster adoption and benefit from economies of scale.</p>
<h2>Conclusion</h2>
<p>All this said, we accomplished a lot with what tiny resources we had. Ok.com brought some amazing people who contributed a lot of reviews and ratings. It was a tremendous amount of fun, as always, to build something and see people using it. Watching the user-generated content come in was a blast and the conversations that occurred around movies and games was always a joy.</p>
<p>Not bad for such a small team on a tiny budget.</p>
GitHub Terminal Cheatsheet2017-07-07T00:00:00Zhttps://www.robkjohnson.com/posts/github-terminal-cheatsheet/<h2>Setting up git on a remote server</h2>
<p>When setting up git on a server for the first time, first initialize git in the root directory of your project. E.g. <code>/var/www/html</code>. I prefer to initialize git before adding any source code and the instructions below reflect this.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token comment"># then initialize the git repo</span></span>
<span class="highlight-line"><span class="token function">git</span> init</span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token comment"># now clone the project from your git repo into the directory</span></span>
<span class="highlight-line"><span class="token function">git</span> remote <span class="token function">add</span> origin git@github.com:username/repository</span></code></pre>
<h2>Check to see if GitHub has your SSH key</h2>
<p>Check to see if your machine has SSH keys stored on GitHub. It will attempt to ssh to GitHub Enterprise</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">ssh</span> <span class="token parameter variable">-T</span> git@hostname</span></code></pre>
<p>You should see something like this if you already have an SSH ket installed</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">The authenticity of <span class="token function">host</span> <span class="token string">'hostname (192.30.252.1)'</span> can't be established.</span>
<span class="highlight-line">RSA key fingerprint is <span class="token number">16</span>:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.</span>
<span class="highlight-line">Are you sure you want to <span class="token builtin class-name">continue</span> connecting <span class="token punctuation">(</span>yes/no<span class="token punctuation">)</span>?</span></code></pre>
<p>Type <code>yes</code> and you should see</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">Hi username<span class="token operator">!</span> You've successfully authenticated, but GitHub does not provide shell access.</span></code></pre>
A Quick Way to Remove all Local git Branches Except for Master2017-06-26T00:00:00Zhttps://www.robkjohnson.com/posts/remove-all-git-branches-except-master/<p>Below is a quick way to remove all git branches on your local machine, except for master. This is really helpful for those of us who commit new branches for each new task.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">git</span> branch <span class="token operator">|</span> <span class="token function">grep</span> <span class="token parameter variable">-v</span> <span class="token string">"master"</span> <span class="token operator">|</span> <span class="token function">xargs</span> <span class="token function">git</span> branch <span class="token parameter variable">-D</span></span></code></pre>
<p>This snippet was written by <a href="https://twitter.com/adantj">Adan Alvarado</a>.</p>
Find/Replace URL's with Javascript2016-08-08T00:00:00Zhttps://www.robkjohnson.com/posts/find-replace-urls-with-javascript/<p>I recently had a need to ensure all URL's on site A were pointing to site B. What I needed was some javascript that would rewrite the URL's once the page had finished loading, and change any site A links to site B links.</p>
<p>Here's a handy piece of javascript that <a href="https://twitter.com/n00b2pr0">@n00b2pr0</a> shared with me that get's the job done.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">var</span> <span class="token function-variable function">linkRewriter</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token parameter">a<span class="token punctuation">,</span> b</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token function">$</span><span class="token punctuation">(</span><span class="token string">'a[href*="'</span> <span class="token operator">+</span> a <span class="token operator">+</span> <span class="token string">'"]'</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">each</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token function">$</span><span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">attr</span><span class="token punctuation">(</span><span class="token string">'href'</span><span class="token punctuation">,</span> <span class="token function">$</span><span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">attr</span><span class="token punctuation">(</span><span class="token string">'href'</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span>a<span class="token punctuation">,</span> b<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token function">linkRewriter</span><span class="token punctuation">(</span><span class="token string">'current-domain.com'</span><span class="token punctuation">,</span> <span class="token string">'desired-domain.com'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
What is a Data Layer?2016-03-16T00:00:00Zhttps://www.robkjohnson.com/posts/what-is-a-data-layer/<p>A <code>dataLayer</code> is an efficient way of presenting multiple data attributes to several services. For example, if you needed to integrate multiple services into your website, such as Google Analytics, Optimizely and Olark, you could have each of them reference data from the same store of data, or <code>dataLayer</code>, instead of rendering multiple variables with the same values for each service. The idea being that a developer can present the data once and it can be easily passed along to multiple services making new service integrations easy. A tag manager can also grab these variables and pass them along to any third-party service also.</p>
<blockquote>
<p>A <code>dataLayer</code> is a javascript variable array that can hold custom information which can be easily accessed by other services. This data can originate from an event such as a page view or a click event. It's a great way to standardize your data for tracking purposes and, once setup, can be managed without development/IT resources.</p>
</blockquote>
<p>Here's how the <code>dataLayer</code> is initiated:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">dataLayer <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span></code></pre>
<p>Here's a <code>dataLayer</code> that has some data in it:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">dataLayer <span class="token operator">=</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'userId'</span><span class="token operator">:</span> <span class="token string">'12345'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'visitorType'</span><span class="token operator">:</span> <span class="token string">'high-value'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'siteSection'</span><span class="token operator">:</span> <span class="token string">'computers'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'product'</span><span class="token operator">:</span> <span class="token string">'MacBook Pro 15 Inch'</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>Here's an example of how to access the value of the variable called <code>visitorType</code> referenced above via javascript.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">dataLayer<span class="token punctuation">.</span>visitorType</span></code></pre>
<p>The above code is referencing the first array via <code>[0]</code>. There can be many arrays compositing the <code>dataLayer</code> and traversing between them is as simple as defining this value.</p>
<p>One of the issues above is that we're mixing different types of data into the same array. For example, I'd prefer to keep customer/global data in the first array and then page/product specific data in the second array. Here's An example of the prior <code>dataLayer</code> but with multiple arrays.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">dataLayer <span class="token operator">=</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'customerInfo'</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'userId'</span><span class="token operator">:</span> <span class="token string">'12345'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'visitorType'</span><span class="token operator">:</span> <span class="token string">'high-value'</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"> <span class="token string-property property">'pageInfo'</span><span class="token operator">:</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token string-property property">'siteSection'</span><span class="token operator">:</span> <span class="token string">'computers'</span><span class="token punctuation">,</span></span>
<span class="highlight-line"> <span class="token string-property property">'product'</span><span class="token operator">:</span> <span class="token string">'MacBook Pro 15 Inch'</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>Here's how you can extract the value of userId from the above example:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">dataLayer<span class="token punctuation">.</span>customerInfo<span class="token punctuation">.</span>userId</span></code></pre>
<p>The above command is accessing the first array, which is the dataLayer, then requesting the variable <code>customerInfo</code> which is also an array, so we request the first array via <code>[0]</code> and then the variable <code>userId</code> which returns the value <code>12345</code>.</p>
<h2>Summary</h2>
<p>Essentially the <code>dataLayer</code> is nothing more than a javascript variable that contains an array. The advantage of doing this though is that new services are implementing standardized features that target this variable and its contents.</p>
<p>There are no predefined rules for what data should be included in a <code>dataLayer</code> and so it's an open book depending on what your needs are. This makes things extremely flexible, but also partially confusing if you don't know where to start.</p>
<p>Resources:</p>
<ul>
<li><a href="https://developers.google.com/tag-manager/devguide#datalayer">Google Tag Manager Guide</a></li>
<li><a href="http://www.simoahava.com/analytics/data-layer/">SimoHava.com</a></li>
</ul>
Create a Better Workflow To Capture & Organize Screenshots with Alfred for Mac2015-10-08T00:00:00Zhttps://www.robkjohnson.com/posts/a-better-screenshot-workflow-using-alfred/<blockquote>
<p>Update: Looks like an update that came out in August 2019 requires a redo of this process. Added another step to remove the screenshot delay caused by the screenshot floating thumbnail. See new optional last step.</p>
</blockquote>
<p>If you find yourself taking copious amounts of screenshots on a Mac, you'll often find yourself with a desktop full of screenshot files. Not only that, but if you order your files by name and kind, you'll find that your most recent screenshots aren't always at the bottom of the list of files. Here's a workflow that has helped me using <a href="https://www.alfredapp.com/">Alfred</a>.</p>
<h2>End Result: Type "SS " and get a quick list of your most recent screenshots</h2>
<p><img src="https://www.robkjohnson.com/assets/img/content/screenshot-workflow.gif" alt="Recent Screenshots Demo" /></p>
<p>You can optionally continue to search by date. Hitting enter on any of these results will reveal it in Finder.</p>
<hr />
<h2>3 Step How-to</h2>
<h3>1. Create a new folder for your screenshots</h3>
<p>First, create a folder on your mac where you'd like to store all of your screenshots. I've created a folder called "Screenshots" in the same directory as "Documents", "Music", Movies" etc folders.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/screenshot-folder-location.png" alt="Screenshots folder location" title="screenshots folder location on mac" /></p>
<h3>2. Change the default location of where your screenshots get stored by entering the following commands in Terminal</h3>
<p>Then open the Terminal app and enter in the text below and hit enter</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">defaults <span class="token function">write</span> com.apple.screencapture location ~/Screenshots/</span></code></pre>
<p>I like to put a name to my screenshots. To do this replace "Rob Johnson" with your name and paste that command line into terminal and hit enter.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">defaults <span class="token function">write</span> com.apple.screencapture name <span class="token string">"Rob Johnson"</span></span></code></pre>
<p>Now you just need to restart SystemUIServer to have the changes take effect.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">killall</span> SystemUIServer</span></code></pre>
<h3>3. Download and add the Screenshot Workflow for Alfred</h3>
<p>I've adapted a workflow that was created by <a href="http://vitorgalvao.com/">Vítor Galvão</a>, that lists the most recent download files in the "Screenshot" directory in descending order. You can also search for a specific date if you need to. For example, if I wanted to search for screenshots I took on Feb 21, I would type "SS 2016-02-21".</p>
<p><a href="https://www.robkjohnson.com/assets/files/Recent-Screenshots.alfredworkflow" class="button download" id="Alfred Screenshot Workflow"><span class="icon is-danger"><strong>▼</strong></span>Download Workflow</a></p>
<h3>4. Remove Delay from Screenshot Capture (optional)</h3>
<p>Mac OSX has a new feature where you can markup your screenshots. Pretty great, but it does cause a delay in the screenshot appearing in your screenshot folder as it waits to see if you want to edit it before it saves.</p>
<p>To turn this off open the "Screenshot" app and deselect the option "Show Floating Thumbnail."</p>
<p>End result: Quickly retrievable screenshots that don't clutter your desktop! Hopefully this will help a few folks. If you have any feedback, you can contact me <a href="https://twitter.com/robkellas">@robkellas</a>.</p>
How to Set & Read Javascript Cookies2015-09-05T00:00:00Zhttps://www.robkjohnson.com/posts/how-to-set-and-read-javascript-cookies/<p>Here are a few tricks with javascript that can be adapted for many uses. That said, the users browser will need to have javascript enabled. The following code doesn't require jQuery.</p>
<h2>Step 1: Create a cookie</h2>
<pre class="language-js"><code class="language-js"><span class="highlight-line">document<span class="token punctuation">.</span>cookie<span class="token operator">=</span><span class="token string">"cookiename=cookievalue"</span><span class="token punctuation">;</span></span></code></pre>
<h2>Step 2: Initial script to allow you to read cookies</h2>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">var</span> cookies<span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">function</span> <span class="token function">readCookie</span><span class="token punctuation">(</span><span class="token parameter">name<span class="token punctuation">,</span>c<span class="token punctuation">,</span><span class="token constant">C</span><span class="token punctuation">,</span>i</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">if</span><span class="token punctuation">(</span>cookies<span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">return</span> cookies<span class="token punctuation">[</span>name<span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> c <span class="token operator">=</span> document<span class="token punctuation">.</span>cookie<span class="token punctuation">.</span><span class="token function">split</span><span class="token punctuation">(</span><span class="token string">'; '</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> cookies <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token keyword">for</span><span class="token punctuation">(</span>i<span class="token operator">=</span>c<span class="token punctuation">.</span>length<span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span> i<span class="token operator">>=</span><span class="token number">0</span><span class="token punctuation">;</span> i<span class="token operator">--</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token constant">C</span> <span class="token operator">=</span> c<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">split</span><span class="token punctuation">(</span><span class="token string">'='</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> cookies<span class="token punctuation">[</span><span class="token constant">C</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token constant">C</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token keyword">return</span> cookies<span class="token punctuation">[</span>name<span class="token punctuation">]</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line">window<span class="token punctuation">.</span>readCookie <span class="token operator">=</span> readCookie<span class="token punctuation">;</span> <span class="token comment">// or expose it however you want</span></span></code></pre>
<h2>Step 3: Request the value of a specific cookie</h2>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token function">readCookie</span><span class="token punctuation">(</span><span class="token string">"cookiename"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
Top 5 Mistakes I Made Replatforming a Multimillion Dollar Ecommerce Website2015-09-02T00:00:00Zhttps://www.robkjohnson.com/posts/5-mistakes-i-made-replatforming/<p>I gave this presentation at the <a href="https://www.slideshare.net/robkellas/top-5-mistakes-i-made-replatforming-a-multimillion-dollar-ecommerce-website">2015 SLC|SEM Digital Marketing Conference</a>. I'm always grateful to have the opportunity to speak at conferences and the folks who run SLC|SEM are great people.</p>
<p>The deck outlines some of the biggest mistakes I felt I made/contributed to, during the recent <a href="https://deseretbook.com/">DeseretBook.com</a> replatforming project that has consumed me for the past year.</p>
<h2>Mistakes</h2>
<ol>
<li>No Dedicated QA or stable staging environment</li>
<li>Performance / load testing was not performed on checkout</li>
<li>Marketing team didn't have a good understanding of the new promotions engine</li>
<li>Didn't obsess over old external and internal links</li>
<li>Didn't understand the SEO impact when switching from http to https</li>
</ol>
<p>See the original presentation below.</p>
<hr />
<iframe src="https://www.slideshare.net/slideshow/embed_code/key/o0fZgkBmXyHOs9" width="425" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen=""> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/robkellas/top-5-mistakes-i-made-replatforming-a-multimillion-dollar-ecommerce-website" title="Top 5 mistakes I made replatforming a multimillion dollar eCommerce website" target="_blank">Top 5 mistakes I made replatforming a multimillion dollar eCommerce website</a> </strong> from <strong><a href="https://www.slideshare.net/robkellas" target="_blank">Rob Johnson</a></strong> </div>
<p>Some tweets from the conference:</p>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">When changing ecommerce platforms you need to create a culture around revenue impact. <a href="https://twitter.com/robkellas">@robkellas</a> <a href="https://twitter.com/hashtag/slcsem?src=hash">#slcsem</a> <a href="https://twitter.com/hashtag/dmc2015?src=hash">#dmc2015</a></p>— SLC SEM Prof Org (@slcsem) <a href="https://twitter.com/slcsem/status/639171793664667648">September 2, 2015</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">If you want to flourish, you have to understand the systems people use & how they’re using them to accomplish a goal <a href="https://twitter.com/DuaneForrester">@DuaneForrester</a> <a href="https://twitter.com/hashtag/SLCSEM?src=hash">#SLCSEM</a></p>— SLC SEM Prof Org (@slcsem) <a href="https://twitter.com/slcsem/status/639202439695085568">September 2, 2015</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">People don't hire you to work. They hire you for results. <a href="https://twitter.com/RickGalan">@RickGalan</a> <a href="https://twitter.com/hashtag/slcsem?src=hash">#slcsem</a></p>— Rob Johnson (@robkellas) <a href="https://twitter.com/robkellas/status/639128148664172544">September 2, 2015</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Put equal time in distributing content as creating content <a href="https://twitter.com/hashtag/SLCSEM?src=hash">#SLCSEM</a> <a href="https://twitter.com/hashtag/dmc2015?src=hash">#dmc2015</a> <a href="https://twitter.com/JordanKasteler">@JordanKasteler</a></p>— Dan Bischoff (@DanBischoff) <a href="https://twitter.com/DanBischoff/status/639118906439761920">September 2, 2015</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Ownership = Making promises + Seeing them through <a href="https://twitter.com/hashtag/slcsem?src=hash">#slcsem</a> via <a href="https://twitter.com/RickGalan">@RickGalan</a></p>— Robert Brady (@robert_brady) <a href="https://twitter.com/robert_brady/status/639132645125918720">September 2, 2015</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Not an empty seat in the house! As expected, <a href="https://twitter.com/CyrusShepard">@cyrusshepard</a> is killing it at <a href="https://twitter.com/hashtag/slcsem?src=hash">#slcsem</a> this morning! <a href="https://twitter.com/hashtag/DMC2015?src=hash">#DMC2015</a> <a href="http://t.co/wCjCgdUtGd">pic.twitter.com/wCjCgdUtGd</a></p>— Ryan Nadeau (@dealership) <a href="https://twitter.com/dealership/status/639097666731806720">September 2, 2015</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
Javascript DOM Manipulation & Other Tricks2015-09-01T00:00:00Zhttps://www.robkjohnson.com/posts/javascript-dom-manipulation/<p>Some handy JavaScript to manipulate a webpage.</p>
<h2>If URL param value is (x) then change the background image</h2>
<p>The following code grabs the parameters of the URL, targets a parameter called "utm_content" and checks to see if the value is "e9150826b" and then adds a background image to the body if true.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">getURLParameter</span><span class="token punctuation">(</span><span class="token parameter">name</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">decodeURIComponent</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">RegExp</span><span class="token punctuation">(</span><span class="token string">'[?|&]'</span> <span class="token operator">+</span> name <span class="token operator">+</span> <span class="token string">'='</span> <span class="token operator">+</span> <span class="token string">'([^&;]+?)(&|#|;|$)'</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">exec</span><span class="token punctuation">(</span>location<span class="token punctuation">.</span>search<span class="token punctuation">)</span><span class="token operator">||</span><span class="token punctuation">[</span><span class="token punctuation">,</span><span class="token string">""</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex">\+</span><span class="token regex-delimiter">/</span><span class="token regex-flags">g</span></span><span class="token punctuation">,</span> <span class="token string">'%20'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token operator">||</span><span class="token keyword">null</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"></span>
<span class="highlight-line">targetVar <span class="token operator">=</span> <span class="token function">getURLParameter</span><span class="token punctuation">(</span><span class="token string">'utm_content'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">if</span> <span class="token punctuation">(</span>targetVar <span class="token operator">==</span> <span class="token string">'e9150826b'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">'body'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>style<span class="token punctuation">.</span>background <span class="token operator">=</span> <span class="token string">'url(/assets/img/content/stars2.jpg)'</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Applications for this include A/B testing a campaign by changing the "utm-content" variable. In the case above "e9150826b" would display the background image, where any other utm_content variable wouldn't initiate anything (control). You can test this now by clicking: <a href="https://www.robkjohnson.com/posts/javascript-dom-manipulation?param=test1">/posts/javascript-dom-manipulation?param=test1</a>. There are a number of various applications in using this. For example you could load up and change multiple elements on the page when this variable is triggered in the URL. The same could be applied by using a cookie method instead of a URL param. However, I thought it is quite an interesting idea to base it off a Google Campaign URL, as it will auto-track results in Google Analytics. Play around by changing the param in the URL, it will only trigger when the parameter "param" is set to "test1".</p>
<hr />
<h2>Change DOM element with javascript</h2>
<p>Sometimes when I'm in a pinch I use Google Tag Manager to change a DOM element by using a javascript tag.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">'article'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token string">"<h1>Wow, now you've gone and done it! Where did all the content go?</h1><p><a href='/javascript-dom-manipulation'>click here to get back to the post</a></p>"</span><span class="token punctuation">;</span></span></code></pre>
<p>I'm going to add this to the code block I created above and change it so that it will only initiate where the "param" value is "test2". <a href="https://www.robkjohnson.com/posts/javascript-dom-manipulation?param=test2">/posts/javascript-dom-manipulation?param=test2</a>.</p>
<p>So now my javascript looks like this:</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">getURLParameter</span><span class="token punctuation">(</span><span class="token parameter">name</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">decodeURIComponent</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">RegExp</span><span class="token punctuation">(</span><span class="token string">'[?|&]'</span> <span class="token operator">+</span> name <span class="token operator">+</span> <span class="token string">'='</span> <span class="token operator">+</span> <span class="token string">'([^&;]+?)(&|#|;|$)'</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">exec</span><span class="token punctuation">(</span>location<span class="token punctuation">.</span>search<span class="token punctuation">)</span><span class="token operator">||</span><span class="token punctuation">[</span><span class="token punctuation">,</span><span class="token string">""</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex">\+</span><span class="token regex-delimiter">/</span><span class="token regex-flags">g</span></span><span class="token punctuation">,</span> <span class="token string">'%20'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token operator">||</span><span class="token keyword">null</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line">targetVar <span class="token operator">=</span> <span class="token function">getURLParameter</span><span class="token punctuation">(</span><span class="token string">'param'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">if</span> <span class="token punctuation">(</span>targetVar <span class="token operator">==</span> <span class="token string">'test1'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">'body'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>style<span class="token punctuation">.</span>background <span class="token operator">=</span> <span class="token string">'url(/assets/img/content/green-bg.jpg)'</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token keyword">if</span> <span class="token punctuation">(</span>targetVar <span class="token operator">==</span> <span class="token string">'test2'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">'body'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>style<span class="token punctuation">.</span>background <span class="token operator">=</span> <span class="token string">'url(/assets/img/content/stars2.jpg)'</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">getElementsByTagName</span><span class="token punctuation">(</span><span class="token string">'article'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token string">"Wow, now you've gone and done it! Where did all the content go?<a href='/javascript-dom-manipulation'>click here to get back to the post</a>"</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>You can also target CSS ID's using the following:</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">'name-of-id'</span><span class="token punctuation">)</span><span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token string">'you could change in image, text etc'</span><span class="token punctuation">;</span></span></code></pre>
<p>Or alternatively target a specific class:</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">document<span class="token punctuation">.</span><span class="token function">getElementsByClassName</span><span class="token punctuation">(</span><span class="token string">'name-of-class'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token string">'you could change in image, text etc'</span><span class="token punctuation">;</span></span></code></pre>
<p>Note that in the case of a class, you can specify a specific instance of the class by changing the <code>[0]</code> to if the javascript should target the first, second, third etc instance of the class. Remember that 0, in this case refers to the first instance of the class.</p>
<hr />
<h2>Create a DOM element via Javascript</h2>
<p>Sometimes you may want to create a dom element in javascript from scratch.</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token keyword">var</span> newDiv <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"div"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line">newDiv<span class="token punctuation">.</span>id <span class="token operator">=</span> <span class="token string">"div1"</span><span class="token punctuation">;</span></span>
<span class="highlight-line">newDiv<span class="token punctuation">.</span><span class="token function">setAttribute</span><span class="token punctuation">(</span><span class="token string">"data-name"</span><span class="token punctuation">,</span> <span class="token string">"Rob Johnson"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line">newDiv<span class="token punctuation">.</span>style<span class="token punctuation">.</span>margin <span class="token operator">=</span> <span class="token string">"0px auto"</span><span class="token punctuation">;</span></span>
<span class="highlight-line">newDiv<span class="token punctuation">.</span>style<span class="token punctuation">.</span>textAlign <span class="token operator">=</span> <span class="token string">"center"</span><span class="token punctuation">;</span></span>
<span class="highlight-line">newDiv<span class="token punctuation">.</span>className <span class="token operator">=</span> <span class="token string">"setclass"</span><span class="token punctuation">;</span></span>
<span class="highlight-line">newDiv<span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token string">"<a href='#'>You can enter any HTML here</a>"</span><span class="token punctuation">;</span></span>
<span class="highlight-line">document<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>newDiv<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<hr />
<h2>Another way to create a DOM element in Javascript</h2>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// create element</span></span>
<span class="highlight-line"><span class="token keyword">var</span> newElement <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"div"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token comment">// style this new element</span></span>
<span class="highlight-line">newElement<span class="token punctuation">.</span>style<span class="token punctuation">.</span>cssText <span class="token operator">=</span> <span class="token string">"margin: 20px auto; max-width: 600px;"</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token comment">// add class name</span></span>
<span class="highlight-line">newElement<span class="token punctuation">.</span>className <span class="token operator">=</span> <span class="token string">"columns is-centered"</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token comment">// define the innerHTML of the new div</span></span>
<span class="highlight-line">newElement<span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token string">"<a href='/destination-url'><img src='/img/my-super-awesome-image.jpg'></a>"</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token comment">// selet the element on the page</span></span>
<span class="highlight-line"><span class="token keyword">var</span> targetElement <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">querySelector</span><span class="token punctuation">(</span><span class="token string">'.class_of_target_div'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token comment">// DO IT</span></span>
<span class="highlight-line">targetElement<span class="token punctuation">.</span>parentNode<span class="token punctuation">.</span><span class="token function">insertBefore</span><span class="token punctuation">(</span>newElement<span class="token punctuation">,</span> targetElement<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>You can see above, that I'm also styling the object with <code>newElement.style.cssText</code>. There are many <code>.style</code> functions you can use to style up your HTML element. You can find a great list of all <a href="https://www.w3schools.com/jsref/dom_obj_style.asp">javascript .style options here</a>.</p>
<hr />
<h2>Redirect page</h2>
<p>Simple method of redirecting the browser</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">window<span class="token punctuation">.</span>location <span class="token operator">=</span> <span class="token string">"https://google.com"</span><span class="token punctuation">;</span></span></code></pre>
<script>
function getURLParameter(name) {
return decodeURIComponent((new RegExp('[?|&]' + name + '=' + '([^&;]+?)(&|#|;|$)').exec(location.search)||[,""])[1].replace(/\+/g, '%20'))||null;}
targetVar = getURLParameter('param');
if (targetVar == 'test1') {
document.getElementsByTagName('body')[0].style.background = 'url(/assets/img/content/green-bg.jpg)';
}
if (targetVar == 'test2') {
document.getElementsByTagName('body')[0].style.background = 'url(/assets/img/content/stars2.jpg)';
document.getElementsByClass('content')[0].innerHTML = "<h1>Wow, now you've gone and done it! Where did all the content go?</h1><p><a href='/javascript-dom-manipulation'>click here to get back to the post</a></p>";
}
</script>
Optimizing Responsive Websites for Users & Search Engines2015-02-25T00:00:00Zhttps://www.robkjohnson.com/posts/optimizing-responsive-websites-for-users-and-search-engines/<p>I gave this presentation at Intersect 3.0 in Salt Lake City. The deck outlines some of the methods you can optimize responsive design for both users and search engines.</p>
<p>If you have any questions or feedback, feel free to ping me via Twitter <a href="https://twitter.com/robkellas">@robkellas</a>.</p>
<iframe src="https://www.slideshare.net/slideshow/embed_code/key/3357xfmNG0bsRm" width="100%" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" allowfullscreen=""> </iframe> <div style="margin-bottom:5px"> <strong> <a href="https://www.slideshare.net/robkellas/optimizing-responsive-websites-for-users-and-search-engines" title="Optimizing Responsive Websites for Users and Search Engines " target="_blank">Optimizing Responsive Websites for Users and Search Engines </a> </strong> from <strong><a href="https://www.slideshare.net/robkellas" target="_blank">Rob Johnson</a></strong> </div>
How to Solve the Fizz Buzz Code Challenge2014-10-06T00:00:00Zhttps://www.robkjohnson.com/posts/fizz-buzz-code-challenge/<p>Below is a common problem that has been used to test developer candidates during job interviews. It's a good problem to review and check to see if you understand some of the basic javascript concepts. Heads up, this is probably very old and not likely asked anymore in 2020 :)</p>
<h2>Problem</h2>
<blockquote>
<p>Write a program that prints the numbers from 1 to 100. But for multiples of three print 'Fizz' instead of the number and for the multiples of five print 'Buzz'. For numbers which are multiples of both three and five print 'FizzBuzz'.</p>
</blockquote>
<h2>Helpful Resources</h2>
<ol>
<li>Fizz Buzz tutorial via <a href="http://www.c2.com/cgi/wiki?FizzBuzzTest">c2.com</a></li>
<li>The FizzBuzz Kata, <a href="https://www.youtube.com/watch?v=CHTep2zQVAc">using Ruby</a></li>
</ol>
<hr />
<h2>Answer</h2>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// Start the count from 1. Limit to 100.</span></span>
<span class="highlight-line"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span> i <span class="token operator"><=</span> <span class="token number">100</span><span class="token punctuation">;</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"> <span class="token comment">// Define write conditions based off divisibility</span></span>
<span class="highlight-line"> document<span class="token punctuation">.</span><span class="token function">write</span><span class="token punctuation">(</span><span class="token punctuation">(</span>i <span class="token operator">%</span> <span class="token number">3</span> <span class="token operator">?</span> <span class="token string">""</span> <span class="token operator">:</span> <span class="token string">"Fizz"</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>i <span class="token operator">%</span> <span class="token number">5</span> <span class="token operator">?</span> <span class="token string">""</span> <span class="token operator">:</span> <span class="token string">"Buzz"</span><span class="token punctuation">)</span> <span class="token operator">||</span> i<span class="token punctuation">)</span></span>
<span class="highlight-line"> <span class="token comment">// Add line breaks for display purposes</span></span>
<span class="highlight-line"> <span class="token operator">+</span> document<span class="token punctuation">.</span><span class="token function">write</span><span class="token punctuation">(</span><span class="token string">"<br>"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span>
<span class="highlight-line"> <span class="token comment">//Increment i for next loop</span></span>
<span class="highlight-line"> i<span class="token operator">++</span><span class="token punctuation">;</span></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<h2>Example output of the above code</h2>
<pre class="language-text"><code class="language-text"><span class="highlight-line">1</span>
<span class="highlight-line">2</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">4</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">7</span>
<span class="highlight-line">8</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">11</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">13</span>
<span class="highlight-line">14</span>
<span class="highlight-line">FizzBuzz</span>
<span class="highlight-line">16</span>
<span class="highlight-line">17</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">19</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">22</span>
<span class="highlight-line">23</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">26</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">28</span>
<span class="highlight-line">29</span>
<span class="highlight-line">FizzBuzz</span>
<span class="highlight-line">31</span>
<span class="highlight-line">32</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">34</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">37</span>
<span class="highlight-line">38</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">41</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">43</span>
<span class="highlight-line">44</span>
<span class="highlight-line">FizzBuzz</span>
<span class="highlight-line">46</span>
<span class="highlight-line">47</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">49</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">52</span>
<span class="highlight-line">53</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">56</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">58</span>
<span class="highlight-line">59</span>
<span class="highlight-line">FizzBuzz</span>
<span class="highlight-line">61</span>
<span class="highlight-line">62</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">64</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">67</span>
<span class="highlight-line">68</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">71</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">73</span>
<span class="highlight-line">74</span>
<span class="highlight-line">FizzBuzz</span>
<span class="highlight-line">76</span>
<span class="highlight-line">77</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">79</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">82</span>
<span class="highlight-line">83</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">86</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">88</span>
<span class="highlight-line">89</span>
<span class="highlight-line">FizzBuzz</span>
<span class="highlight-line">91</span>
<span class="highlight-line">92</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">94</span>
<span class="highlight-line">Buzz</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">97</span>
<span class="highlight-line">98</span>
<span class="highlight-line">Fizz</span>
<span class="highlight-line">Buzz</span></code></pre>
How to use Macros in Google Tag Manager2014-05-18T00:00:00Zhttps://www.robkjohnson.com/posts/how-to-use-macros-in-google-tag-manager/<div class="callout"><span class="is-info">Update: </span> Google Tag Manager has since changed the name "Macros", to "Variables". There is now a section, similar to tags and triggers, called variables where you can manage these variables and call them into your tags.</div>
<p>I’ve been using <a href="http://google.com/tagmanager">Google Tag Manager</a> for a number of months and have been thoroughly enjoying deploying tag changes on my sites instantly without having wait for a production deploy. Also, having the power to select what tags fire on what pages is a huge plus for me. It removes the need for development resources and gives me greater control in the delivery of my tags; all of which are delivered asynchronously. Pretty smart.</p>
<p>One tool available in Google Tag Manager, that I’ve found extremely useful recently, are macros. <a href="http://en.wikipedia.org/wiki/Macro_computer_science">Macros can mean and be used in a lot of different ways</a> and so, initially, I was a little confused as to how macros could be used in Google Tag Manager.</p>
<p>Below is one of the many use cases that illustrates how you can use macros to minimize code on your site and deliver much greater functionality to your web applications.</p>
<h2>Use Case: Deliver a unique offer to users who click through to a page via an email campaign</h2>
<p>If you’re using Google Analytics and sending emails to customers. Chances are you’re already including <strong>utm_campaign</strong> in your email URL’s to track your campaigns. An example URL may be</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line">https://example.com/example-product?&utm_source=example.com&utm_medium=email&utm_campaign=abandon_cart</span></code></pre>
<p>To grab the value of the query key or parameter create a new macro in GTM with the macro type as <strong>URL</strong> and the component type as <strong>Query</strong>. Then enter in the query key which, in this example, is</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">utm_campaign</span></code></pre>
<p><img src="https://www.robkjohnson.com/assets/img/content/macros-google-tag-manager-example.png" alt="Macros in Google Tag Manager" /></p>
<p>This will then assign the value of <strong>utm_campaign</strong> from the URL, which in this case will be <strong>abandon_cart</strong>, to the macro</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token punctuation">{</span><span class="token punctuation">{</span> campaign_name <span class="token punctuation">}</span><span class="token punctuation">}</span></span></code></pre>
<p>This macro value will now be available as a variable for code on any page where the parameter exists in the URL. Nifty.</p>
<h2>Pass the macro into the code</h2>
<p>We can then initiate a popup modal via javascript with a simple check</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">if</span> <span class="token punctuation">(</span> <span class="token punctuation">{</span><span class="token punctuation">{</span> campaign_name <span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token operator">==</span> <span class="token string">"abandon_cart"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token comment">// offer code for popup modal</span></span>
<span class="highlight-line"></span>
<span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>There are lots of other use cases that include passing these variables into other tracking services etc. The great thing about GTM is that it enables you to be creative with minimal development knowledge.</p>
<p>Feel free to reach out via <a href="http://twitter.com/robkellas" title="@robkellas">Twitter</a> if you have any questions.</p>
Turning on Server Errors for MAMP2014-03-06T00:00:00Zhttps://www.robkjohnson.com/posts/mamp-turn-server-errors-on/<p>Debugging your apps can be a moneumental pain without proper logging or error reports. Mamp ships with errors turned off. Here is how you turn them back on.</p>
<p>Find out what version of PHP you are running on MAMP. You can find this by going to the MAMP preferences pain. Here you can see I'm using <strong>5.5.3</strong></p>
<p><img src="https://www.robkjohnson.com/assets/img/content/mamp-preferences.jpg" alt="MAMP preferences pain" /></p>
<p>Now check your php.ini located at</p>
<pre class="language-ini"><code class="language-ini"><span class="highlight-line">Applications/MAMP/bin/php/**php5.5.3**/conf/php.ini</span></code></pre>
<p>Open it up in a text editor and search for <code>display_errors</code></p>
<p>You are most likely seeing them turned "Off." Replace "Off" with "On" and you should be set.</p>
<p>Also, it's worth mentioning that you may have a ";" at the beginning of the line like so. ";" comments out lines which will set these preferences to defaults. What you should be ending up with is something like this.</p>
<pre class="language-ini"><code class="language-ini"><span class="highlight-line"><span class="token comment">; Print out errors (as a part of the output). For production web sites,</span></span>
<span class="highlight-line"><span class="token comment">; you're strongly encouraged to turn this feature off, and use error logging</span></span>
<span class="highlight-line"><span class="token comment">; instead (see below). Keeping display_errors enabled on a production web site</span></span>
<span class="highlight-line"><span class="token comment">; may reveal security information to end users, such as file paths on your Web</span></span>
<span class="highlight-line"><span class="token comment">; server, your database schema or other information.</span></span>
<span class="highlight-line"><span class="token key attr-name">display_errors</span> <span class="token punctuation">=</span> <span class="token value attr-value">On</span></span></code></pre>
<p>Now just restart your MAMP server and you should be good to go.</p>
Spree Conf NYC 2014 Notes2014-03-03T00:00:00Zhttps://www.robkjohnson.com/posts/spreeconf-2014/<p>The company I work for is currently looking at alternative solutions for an Ecommerce platform. Currently we are using a custom solution that is brittle from which we incur a lot of maintenance cost. Some of our shopping requisites for a potential new platform include the avoidance of: hefty monthly license fees, pay-per-order models, dependence on third-party developers for feature development. Yet maintain our current feature set and ideally stay within the rails environment. We're not picky at all...</p>
<h2>Looking for a "solved problem" solution</h2>
<p>Spree has been growing for a number of years in popularity and claims to power <a href="http://spreecommerce.com/">45,000 online stores</a>. With recent news that it has received a <a href="http://venturebeat.com/2014/02/25/spree-commerce-raises-5-million-for-open-source-e-tailer-software/">$5 million in its latest round of funding</a>, it would be daft to ignore the platform as a viable solution.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/sva-theatre.jpg" alt="SVA Theatre - Location of Spreeconf" /></p>
<p><a href="https://twitter.com/thereddshift">@thereddshift</a> and I attended the latest Spree conference in NYC to meet the Spree community and talk with the developers and companies that were using it. After all we want to ensure that this platform has a bright future and an invested developer community so we don't have to revisit this costly exercise in the near future.</p>
<h2>Top Tweets</h2>
<blockquote class="twitter-tweet" lang="en"><p>For good PR, your company needs a good story. No story = no PR, regardless of what agency you choose. <a href="https://twitter.com/search?q=%23spreeconf&src=hash">#spreeconf</a> <a href="https://twitter.com/search?q=%23AndyDunn&src=hash">#AndyDunn</a></p>— Rob Johnson (@robkellas) <a href="https://twitter.com/robkellas/statuses/438781547957997570">February 26, 2014</a></blockquote>
<blockquote class="twitter-tweet" lang="en"><p>"Right decisions don't necessarily lead to good outcomes" - Jeff Ma at <a href="https://twitter.com/search?q=%23spreeconf&src=hash">#spreeconf</a></p>— Matthew Redd (@mingusredd) <a href="https://twitter.com/mingusredd/statuses/439091621830086656">February 27, 2014</a></blockquote>
<blockquote class="twitter-tweet" lang="en"><p>Amazon has attributed a 29% increase in sales since introducing product recommendation sections throughout the site. <a href="https://twitter.com/search?q=%23spreeconf&src=hash">#spreeconf</a></p>— Rob Johnson (@robkellas) <a href="https://twitter.com/robkellas/statuses/438795568161513473">February 26, 2014</a></blockquote>
<blockquote class="twitter-tweet" lang="en"><p>Page speed matters: Yahoo improved page load time by 300ms and saw 9% increase in rev <a href="https://twitter.com/search?q=%23spreeconf&src=hash">#spreeconf</a></p>— Rob Johnson (@robkellas) <a href="https://twitter.com/robkellas/statuses/438747310663954434">February 26, 2014</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<h2>Day 1 (Wednesday)</h2>
<h3>Sales tax</h3>
<p>Use TaxCloud or AvaTax to calculate tax on orders</p>
<p>Tax is calculated per line item</p>
<p>Difference between tax. Included vs additional</p>
<h3>Adjustments</h3>
<p>Spree will automatically adjust your order to the max value promotion for the customer</p>
<h3>Promotions</h3>
<ul>
<li>You can combine promotions</li>
<li>Not got credits yet</li>
<li>Not got group discounts yet</li>
<li>Supports vat refunding</li>
<li>Almost 2000 test in spree</li>
</ul>
<h3>Spree hub</h3>
<p>A platform to decouple the backend processed from the storefront.</p>
<h3>The Hub</h3>
<p>Working to make it easier
Want a plug and play solution for all platforms</p>
<ul>
<li>Object
<ul>
<li>core of the system customers, carts, addresses etc</li>
<li>each object just requires an ID field.</li>
</ul>
</li>
<li>Event</li>
<li>Webhook</li>
</ul>
<h3>Simplified API</h3>
<blockquote>
<p>it's not about catching demand but creating demand.</p>
</blockquote>
<p>Delighted.com a customer opinion site that shows customer awareness??</p>
<p>Guide shops: build stores that you size up and then they ship to you. Hope to grow to 50 stores within three years.</p>
<hr />
<h2>Day 2 (Thursday)</h2>
<h3>Daniel Honig (railsdog) & Thomas Von Deyen (Alchemy)</h3>
<ul>
<li>
<p>Ecommerce is a hard problem because everyone is doing something different.</p>
</li>
<li>
<p>Grid based ecommerce is over</p>
</li>
<li>
<p>Four pillars of ecommerce</p>
<ol>
<li>Commerce - this is 2014 and everything needs to be integrated. Flexible shopping cart and checkout experience supporting all channels</li>
<li>Content - management systems built for rich storytelling and interactive experience</li>
<li>Community</li>
<li>Context - True personalization of experience, based on location, channel and relationship</li>
</ol>
</li>
</ul>
<h4>Priorities for retailers</h4>
<ul>
<li>Drive excitement for products</li>
<li>Top brands must build a global halo for the brand thorough rich storytelling and address issues of heritage, craftsmanship and cultural relevance</li>
<li>Create an unforgettable shopping experience</li>
</ul>
<h4>Why does storytelling matter</h4>
<p>Customers are using more information than at any other time in history to make purchase decisions</p>
<p>A good storyteller has the power to transport audiences to another world
What I like is to feel emotion... and I agree even more when I see the sales results.</p>
<p>Why do we all like a good story?</p>
<ul>
<li>Escapism</li>
<li>Exploration</li>
<li>Identification</li>
<li>Simulation</li>
<li>Intimacy</li>
<li>Comprehension
<ul>
<li>Stories run on the same hardware that process real events (only the frontal cortex differentiates it)</li>
</ul>
</li>
<li>Resolution</li>
</ul>
<h4>McKenzie Funnel (old model)</h4>
<p>Awareness -> Familiarity -> Consideration -> Purchase -> Loyalty</p>
<h4>McKenzie new model</h4>
<p>Google ZMOT (zero model of truth) - http://www.thinkwithgoogle.com/collections/zero-moment-truth.html</p>
<p>New model:</p>
<ul>
<li>Stimulus</li>
<li>zero moment of truth</li>
<li>point of shelf</li>
<li>purchase experience (missed this)</li>
</ul>
<h4>What's changed</h4>
<ul>
<li>Noise - lots of noise, more than ever</li>
<li>Two-way - social</li>
<li>Consistency/Alignment</li>
<li>Loyalty</li>
</ul>
<p>Another step has been added</p>
<p>Emotion can trigger sales and loyalty</p>
<p>People will remember your story and forget your product</p>
<p>https://alchemy-cms.com/about</p>
<hr />
<h3>Jeff Ma - basis for the main character for the book and movie of 21</h3>
<p>Separate your decision from the outcome. You can make the right decision, and the outcome is irrelevant to whether that decision was correct or not.</p>
<blockquote>
<p>Group think: Making a decision based of consensus of the group in order to avoid conflict</p>
</blockquote>
<p>Innovation requires making unpopular decisions</p>
<blockquote>
<p>Loss aversion: We are more impacted by a loss than a gain of the same amount.</p>
</blockquote>
<blockquote>
<p>Endowment bias: When you become protective of what you have over getting more</p>
</blockquote>
<hr />
<h3></h3>
<blockquote>
<p>Imagine spree as a service using the API.</p>
</blockquote>
<p>Spree can just be a service</p>
<h3>Design and Designers</h3>
<p>Decouple business logic from the experience is the next step</p>
<p>Going 100% JS (client-side)
SEO Implications:
Issues with indexing:</p>
<p>Prerender.io - gem "prerender_rails" every time it knows a bot is requesting a page it will prerender the page via phantom js and spit out the html server side.</p>
<hr />
<h3>godynamo.com</h3>
<ul>
<li>If people are clicking it, make it tappable</li>
<li>Outbound links are a point of no return. Keep people in your site as much as possible.</li>
<li>Instead of saying "cart" show a cart. 20% more clicks on images than words.</li>
<li>Whats your favorite responsive framework? Bootstrap</li>
<li>Why not native site verses a responsive site</li>
<li>A/B testing with Optimizely</li>
</ul>
<hr />
<p>A random pic taken at the top of the Empire State Building.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/empire-state-plaza.jpg" alt="Empire State Plaza, Capitol" /></p>
A Quick Tutorial on Telnet2013-10-06T00:00:00Zhttps://www.robkjohnson.com/posts/a-quick-tutorial-on-telnet/<blockquote>
<p>Telnet is a network protocol used on the Internet or local area networks to provide a bidirectional interactive text-oriented communication facility using a virtual terminal connection. – <em><a href="http://en.wikipedia.org/wiki/Telnet">Wikipedia</a></em></p>
</blockquote>
<h2>Download</h2>
<p>If you don't have telnet installed, install it with this following terminal command:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line"><span class="token function">sudo</span> <span class="token function">apt</span> <span class="token function">install</span> telnet</span></code></pre>
<h2>How to use</h2>
<p>Enter the following lines into terminal</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">telnet robkjohnson.com <span class="token number">80</span></span>
<span class="highlight-line">GET / HTTP/1.1</span>
<span class="highlight-line">host: robkjohnson.com</span></code></pre>
<p>What is returned is essentially plain text of the source code. Which is exactly what your browser receives through the same GET/HTTP1.0 or GET/HTTP1.1 request.</p>
<h2>Security</h2>
<p>Telnet is an old protocol that does not have any security, so anyone can eavesdrop on your connection. So make sure you're not accessing anything private with it.</p>
<p>Thanks goes to <a href="http://twitter.com/xunker">@xunker</a> for teaching me this.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">http://rubular.com <span class="token punctuation">\</span>s-<span class="token punctuation">\</span>s<span class="token punctuation">(</span><span class="token punctuation">\</span>d<span class="token punctuation">{</span><span class="token number">7</span><span class="token punctuation">}</span><span class="token punctuation">)</span></span></code></pre>
Flamingo: A Free Responsive Theme for Get Simple CMS2012-04-07T00:00:00Zhttps://www.robkjohnson.com/posts/flamingo-theme/<p>A while ago I stumbled across the <a href="http://get-simple.info/">GetSimple CMS</a>. At the time I was working on developing my own CMS, which now exists only as a contacts application, because I found that <a href="http://chriscagle.me/">Chris Cagle</a> had done a bunch of work that would save me A LOT of time.</p>
<p>So I've been using it for about a year so far and I thought it was about time that I give back.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/flamingo-screenshot.png" alt="Flamingo Responsive Theme" /></p>
<p><a href="http://get-simple.info/extend/theme/flamingo/410/" class="button download" id="Flamingo Theme" title="Download Flamingo Theme" target="_blank"><span class="icon is-danger"><strong>▼</strong></span>Download Flamingo</a></p>
<h2>Two Themes in One</h2>
<p>Change the css file in inc.start.php from style1.css to style2.css. Style2.css changes the layout to allow for longer nav items. Both styles are as responsive as each other.</p>
<h2>Responsive</h2>
<p>The theme is a responsive design with a <a href="http://stuffandnonsense.co.uk/projects/320andup/">320px and up</a> philosophy. That basically means that if you took out all of the @media calls in the CSS document. It would only render the mobile version. So browsers which do not support @media calls such as, IE 7 and below, will render the mobile version and not dynamically respond to the screen size.</p><p></p>
<h2>HTML5 Boilerplate</h2>
<p>That said, if a user comes to your site in IE6, or another seriously depreciated browser, they will receive an update message notifying that they should update. This is an inherent part of the <a href="http://html5boilerplate.com/">HTML5 Boilerplate</a> that is cooked into this theme. That means <a href="http://www.modernizr.com/">modernizer.js</a> is also part of the theme allowing HTML5 objects to be rendered properly in old browsers.</p>
<h2>Multiple Theme Pages</h2>
<p>This theme comes with some extras that really help it stand out. First of all it comes with the following pages:</p>
<ul>
<li>Template.php (1-3 column layout)</li>
<li>2-column.php (1-2 column layout)</li>
<li>404.php (Set this as your default 404 page at /data/other/404.xml in template field) - Blog.php (Blog tile index page)</li>
</ul>
<h2>Code for image thumbnails</h2>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>ul</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>thumbs<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fbox<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>img</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/data/uploads/blog/flamingo/desert.jpg<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fbox<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>img</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/data/uploads/blog/flamingo/sunflower.jpg<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>a</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fbox<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>img</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/data/uploads/blog/flamingo/jungle.jpg<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>a</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fbox<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>img</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/data/uploads/blog/flamingo/sun.jpg<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>a</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fbox<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>img</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/data/uploads/blog/flamingo/pelicans.jpg<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>a</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>a</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fbox<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>javascript:;<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>img</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/data/uploads/blog/flamingo/landscape.jpg<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>a</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>li</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>ul</span><span class="token punctuation">></span></span></span></code></pre>
<h2>Style2 - add your tagline</h2>
<p>For style2 you need to add a component and label it "tagline". Just enter in the text you wish to appear as your tagline, hit save and it will appear next to the site name you have given for your site (top left - mine is Portfolio & Blog).</p>
<h2>Last but not least, charts</h2>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dl</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>chart<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dd</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>p</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>yellow<span class="token punctuation">"</span></span> <span class="token special-attr"><span class="token attr-name">style</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token value css language-css"><span class="token property">width</span><span class="token punctuation">:</span>92%</span><span class="token punctuation">"</span></span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>span</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>label<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>Illustrator<span class="token tag"><span class="token tag"><span class="token punctuation"><</span>span</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>value<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>92%<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>span</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>span</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>p</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dd</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dd</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>p</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>pink<span class="token punctuation">"</span></span> <span class="token special-attr"><span class="token attr-name">style</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token value css language-css"><span class="token property">width</span><span class="token punctuation">:</span>84%</span><span class="token punctuation">"</span></span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>span</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>label<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>Photoshop<span class="token tag"><span class="token tag"><span class="token punctuation"><</span>span</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>value<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>83%><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>span</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>p</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dd</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dd</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>p</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>yellow<span class="token punctuation">"</span></span> <span class="token special-attr"><span class="token attr-name">style</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token value css language-css"><span class="token property">width</span><span class="token punctuation">:</span>80%</span><span class="token punctuation">"</span></span></span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>span</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>label<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>InDesign<span class="token tag"><span class="token tag"><span class="token punctuation"><</span>span</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>value<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>80%<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>span</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>span</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>p</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dd</span><span class="token punctuation">></span></span></span>
<span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dl</span><span class="token punctuation">></span></span></span></code></pre>
SEO Adoption at Deseret Digital Media2011-11-16T00:00:00Zhttps://www.robkjohnson.com/posts/seo-adoption-at-deseret-digital-media/<p>From 2010-2011 I was very fortunate to work with <a href="https://www.linkedin.com/in/aaronix/">Aaron Wester</a> who at the time was Director over Web Analytics for DDM and knee-deep in understanding the tremendous success and areas of growth for <a href="https://www.ksl.com/">KSL.com</a> and <a href="https://www.deseretbook.com/">DeseretNews.com</a>.</p>
<p>I had developed an SEO ranking document that took the best SEO practices at the time and had a column for development effort. With this I even printed the source code of each main web view of the websites DDM ran, and highlighted each area with the recommended changes. It's funny to think that I once did that.</p>
<p>This was before version control like <a href="https://www.github.com/">GitHub</a> was a thing. Oh those were messy days. Besides I was very early in my career at this time and no sane person trusted me with production credentials quite yet.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/seo-adoption/seo-matrix.png" alt="seo matrix" /></p>
<p class="caption">A screenshot of the seo matrix doc</p>
<p>With this, I also created a "Share of Voice" tracking system. This would analyze the prominence and placement of our content for certain key terms on search engines and would rank each location and provide an overall score in comparison to other news media outlets. This would provide an easy way to view how successful we were being at targeting certain topics in comparison to local competition.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/seo-adoption/share-of-voice.png" alt="seo matrix" /></p>
<p class="caption">A screenshot of the share of voice system I developed</p>
<p>We were able to see incredible gains of search engine referrals to our web properties, with the <a href="https://www.deseretnews.com/">Deseret News</a> seeing an increase in search referrals of 156%.</p>
<p><img src="https://www.robkjohnson.com/assets/img/content/seo-adoption/deseret-news-seo-referral-gains.png" alt="seo matrix" /></p>
<p class="caption">An old Omniture graph of search referrals YoY</p>
<p>You can view the full deck that was presented below.</p>
<p><a href="https://www.slideshare.net/robkellas/seo-adoption-ddm-slcsem-nov-16" class="button download" title="View the deck on slideshare.com" target="_blank"><span class="icon is-danger"><strong>▶</strong></span>View the deck on SlideShare</a></p>
<p>Or download a PDF of the original deck.</p>
<p><a href="https://www.robkjohnson.com/assets/presentations/slc-sem-2011/slc-sem-seo-adoption-at-deseret-digital-media.pdf" class="button download" title="Download a PDF of the deck" target="_blank"><span class="icon is-danger"><strong>▼</strong></span>Download PDF</a></p>