To download this Insights Piece as a printable .PDF, click here.
To watch the full Panel Discussion, click here.
To help define the next steps of your data journey, connect with Databricks’ technologists here.


In response to the volatility erupting from the pandemic, businesses are placing renewed attention on the value of highly-responsive analytical capabilities. This case is particularly pronounced in the Retail arena, where the extreme conditions of the pandemic have provided consumers and retailers alike with a rare opportunity to reassess their relationship. Retail executives are beholden to attract impressionable customers and earn their allegiance as our operating environment corrects. More than ever, national and global Retailers alike are leaning on data analytics to help drive better decision-making and advance customer outcomes, strengthening their businesses against the current volatile economic landscape.

We convened a virtual panel to hear how Darren Abraham, Data Science Manager, JB Hi-Fi, Marcy Larsen, Retail Industry Solutions Executive, Microsoft Australia, and Bryan Smith, Technical Director, Retail & CPG, Databricks are approaching data transformation in this chaotic business environment. Moderated by Bede Hackney, Country Manager, Databricks Australia & NZ, their conversation unearthed the unique strategies and partnerships that Retailers are pursuing to enable responsive, data-informed decision making. We are sharing some of the salient insights from this executive conversation with you today.


Our year has been defined by sheer volatility. Devastating bushfires, a relentless pandemic, civil unrest and renewed advocacy against racial discrimination, and the continuing, systematic erosion of facts on the world stage – the calendar year to date has not shied from issuing historic challenges to our shared social fabric. Huge swathes of the economy were forced into retreat as global manufacturing regions turned cold and the movement of people ceased. Retail sales plummeted in response to these seismic forces, with the global sector projected to decline by 10.5% in 2020.

These conditions can sound cataclysmic. But they can also be alarmist. Our sector outlook shifts noticeably once we account for the tremendous growth in e-commerce facilitated by the nation’s highly responsive Retail community. This is the problem with volatility: it inherently corrupts our ability to forecast with any precision. Our optimism or pessimism on the continued health of our communities and the strength of our industry is not misinformed, it’s simply not informed. We cannot rely on historical data to shape forecast models as we have no baseline from which to gain perspective.

Without historical data to lean on, thought-leaders are taking advantage of new types of information to shape a contextual perspective. The sole attention we once placed on internal data is slackening, our gaze widening to accommodate external data sets and incorporate unstructured and semi-structured sources of data. Now more than ever, we require a holistic data perspective, one that is informed not just by the data we generate, but by bringing the data being generated around us that is relevant to our business operations together in a unified and intuitive platform.


We can uncover an entirely new perspective on the retail value chain by expanding our data thinking outwards to embrace an ecosystems approach. Retailers are confronting their global supply chain dependencies in response to the pandemic, reassessing the weight of individual component costs in the broader scheme of end-to-end value optimisation. But the process of anticipating and mitigating the factors arresting the movement of goods hinges on sharing data with key players in our supply and distribution networks to achieve greater transparency and responsiveness.

Beyond addressing our assumptions on what data is of relevance, we need to consider the varied use cases for data assets we have already collected. Preceding the pandemic, Retailers had been building their capability to collect data from physical environments with the intent of understanding in-store traffic patterns to optimise product placement layouts and ultimately affect cart size. Now, the same environmental data is being directed towards improving employee and customer health and safety protections by removing bottlenecks and managing in-store movement.


Hindsight is the weapon of emerging businesses – you don’t need to encounter and overcome the trials faced by your established competitors when you have the historical data to circumvent them. New companies pre-emptively adopt a data-first mindset, diffusing the resource-draining data engineering undertaken by traditional players.

JB Hi-Fi faced a number of significant challenges on the road to building an intuitive and responsive data functionality. Data had accrued in siloed systems, while different access methods, formats, and authentication controls meant that anyone attempting to report against the data contended with a spider web of connectivity. The on-premises data warehouse solution was both scale and cost prohibitive, while reporting mechanisms anchored to Excel were flat, non-interactive, and non-dynamic in the content they delivered.

In response, the business created an environment structure modelled on swim lanes of data ingestion, one for each type of collection, with these pipelines parametrised to store definitions about schedules, database tables, and credentials. From there, all data is transformed into Databricks Delta Format, allowing for asset level transactions and increased retrieval time.

Microsoft Power BI is augmented with precomputation and business logic implementations within an adjacent Databricks Job to rapidly deliver interactive visualisations and business intelligence reporting capabilities. Together, this environment structure empowers JB HiFi to react rapidly to a changing business landscape.


Dynamic, data-informed decision making is essential for businesses when they encounter volatility. The pandemic has exposed our collective reliance on historical forecasting data, shifting attention towards the value of real-time data to inform rapid responses to changing market forces. Moving forward, Retailers will benefit from looking beyond their own walls to identify and leverage alternative data sources, incorporating structured, semi-structured and unstructured data to reveal broader insights.

By adopting a culture of rapid experimentation and focussing on improving outcomes rather than generating insights, Retailers will be able to unlock a new resilience that is embedded with responsiveness. It is incumbent on Retailers to consider the steps they must take and the partnerships they can pursuit to realise these capabilities without further delay.


Epidemic structural and cultural issues are shackling the potential of General Counsel and commercial firms alike. Institutionalised time-based billing methods are a source of shared frustration, and there is much to be said for how generational change and emerging technologies might affect this highly-strung ecosystem.

Unfortunately, too much of this conversation is being held in whispered tones, which does little to generate impetus for change. Connect Media and VARIO FROM PINSENT MASONS decided to gather leading General Counsel from all sectors of the national economy to spark a constructive and candid dialogue in the hope of creating this change momentum.

Over twenty executives converged for a transparent look at how these different cultural forces and corporate structures interplay, and to consider what must be done to forge a bright new future for the profession as a whole.