(The source code is at https://github.com/jauyeunggithub/bravado-quest)
Sometimes, we want to load large amounts of data in the background in a Nuxt app with web workers
In this article, we’ll look at how to load large amounts of data in the background in a Nuxt app with web workers
How to load large amounts of data in the background in a Nuxt app with web workers?
To run background tasks in a Nuxt app with web workers, we can add the Webpack’s worker-loader package into our Nuxt project.
Also, we’ve to make sure that web workers are only loaded on client side.
We’ll make a project that loads a large JSON file and searches it.
To start, we create the Nuxt project with:
npx create-nuxt-app quest
Then we run:
cd quest
npm run dev
to change to the project folder and run the project.
Next, we add Vuetify into the project with:
npm install @nuxtjs/vuetify -D
And we add the worker-loader package with:
npm i worker-loader@^1.1.1
To store data with IndexedDB, we install the Dexie package.
To install it, we run:
npm i dexie
Next, in nuxt.config.js, we change it to:
export default {
// Global page headers: https://go.nuxtjs.dev/config-head
head: {
title: 'quest',
htmlAttrs: {
lang: 'en',
},
meta: [
{ charset: 'utf-8' },
{ name: 'viewport', content: 'width=device-width, initial-scale=1' },
{ hid: 'description', name: 'description', content: '' },
{ name: 'format-detection', content: 'telephone=no' },
],
link: [{ rel: 'icon', type: 'image/x-icon', href: '/favicon.ico' }],
},
// Global CSS: https://go.nuxtjs.dev/config-css
css: [],
// Plugins to run before rendering page: https://go.nuxtjs.dev/config-plugins
plugins: [
{ src: '~/plugins/inject-ww', ssr: false }
],
// Auto import components: https://go.nuxtjs.dev/config-components
components: true,
// Modules for dev and build (recommended): https://go.nuxtjs.dev/config-modules
buildModules: ['@nuxtjs/vuetify'],
// Modules: https://go.nuxtjs.dev/config-modules
modules: [],
// Build Configuration: https://go.nuxtjs.dev/config-build
mode: 'spa',
build: {
extend(config, { isClient }) {
config.output.globalObject = 'this'
if (isClient) {
config.module.rules.push({
test: /.worker.js$/,
loader: 'worker-loader',
exclude: /(node_modules)/,
})
}
},
},
ssr: false,
}
We added the build.extend method to invoke the worker-loader when any file that ends with .worker.js is found.
isClient makes sure workers only load on client side apps.
config.output.globalObject = 'this' is needed so the hot reloading will work with the web workers loaded.
We also set ssr to false to disable server side rendering.
Also, we add { src: '~/plugins/inject-ww', ssr: false } to add the /plugins/inject-ww to load the worker loading plugin.
We add '@nuxtjs/vuetify' to load Vuetify in Nuxt.
In the plugins folder, we add:
import Worker from '~/assets/js/data.worker.js'
export default (_, inject) => {
inject('worker', {
createWorker () {
return new Worker()
}
})
}
to load the worker from the /assets/js/data.worker.js.
To create worker, we create the data.worker.js file in the /assets/js/ and add:
import db from '@/db'
onmessage = async () => {
if ((await db.avatars.toCollection().toArray()).length > 0) {
postMessage({
loaded: true,
})
return
}
const usersObj = await import(`@/data/users.json`)
const users = Object.values(usersObj)
const usersWithId = users.map((u, id) => ({ ...u, id }))
await db.avatars.bulkPut(usersWithId)
postMessage({
loaded: true,
})
}
@/data/users.json is a big JSON file saved in the project.
We import the users.json file from the data folder and store it in Indexed DB with Dexie.
We can load very large JSON files without crashing the browser since web workers run in a separate thread from the main browser thread.
db comes from db.js, which has:
import Dexie from 'dexie'
const db = new Dexie('db')
db.version(1).stores({
avatars: '++id, address, avatar, city, email, name, title',
})
export default db
We create the db database with the avatars collection with the given keys.
We call bulkPut to write the retrieved data to the collection.
Now we cache the data in Indexed DB so they don’t have to load every time we load the app.
We make sure we try to load the cached data first with:
if ((await db.avatars.toCollection().toArray()).length > 0) {
postMessage({
loaded: true,
})
return
}
And we call postMessage to communicate that loading is done.
The message will be picked by by the message event handler when we invoke this worker.
In the pages, folder, we add _keyword.vue, to add an input and the search results component:
<template>
<v-app>
<v-card width="600px" elevation="0" class="mx-auto">
<v-card-text :elevation="0">
<v-text-field
v-model="keyword"
hide-details
:prepend-inner-icon="mdiMagnify"
full-width
solo
dense
background-color="#FAFAFA"
/>
<SearchResults :keyword="keyword" />
</v-card-text>
</v-card>
</v-app>
</template>
<script >
import { mdiMagnify } from '@mdi/js'
import SearchResults from '@/components/SearchResults'
export default {
components: {
SearchResults,
},
data() {4
return {
keyword: '',
mdiMagnify,
}
},
beforeMount() {
this.keyword = this.$route.params.keyword
},
}
</script>
<style>
html {
overflow: hidden;
}
</style>
Then in components/SearchResults.vue that we created, we add:
<template>
<div>
<div v-if="loading" class="my-3">
<v-card>
<v-card-text> Loading... </v-card-text>
</v-card>
</div>
<div v-else id="scrollable" class="my-3">
<v-card
v-for="r of filteredSearchResultsWithHighlight"
:key="r.id"
class="my-3"
:style="{
border: highlightStatus[r.id] ? '2px solid lightblue' : undefined,
}"
@click="
highlightStatus = {}
$set(highlightStatus, r.id, !highlightStatus[r.id])
"
>
<v-card-text class="d-flex pa-0 ma-0">
<img :src="r.avatar" class="avatar" />
<div
class="flex-grow-1 pa-3 d-flex flex-column justify-space-between right-pane"
>
<div class="d-flex justify-space-between">
<div>
<h2 v-html="r.name"></h2>
<p class="py-0 my-0">
<b v-html="r.title"></b>
</p>
<p class="py-0 my-0">
<span v-html="r.address"></span>,
<span v-html="r.city"></span>
</p>
</div>
<div v-html="r.email"></div>
</div>
<div>
<v-btn text color="#00897B">Mark as Suitable</v-btn>
</div>
</div>
</v-card-text>
</v-card>
</div>
</div>
</template>
<script>
import db from '@/db'
export default {
props: {
keyword: {
type: String,
default: '',
},
},
data() {
return {
highlightStatus: {},
filteredSearchResults: [],
loading: false,
}
},
computed: {
filteredSearchResultsWithHighlight() {
const { keyword } = this
if (!Array.isArray(this.filteredSearchResults)) {
return []
}
const highlighted = this.filteredSearchResults?.map((u) => {
const highlightedEntries = Object.entries(u)?.map(([key, val]) => {
if (key === 'avatar' || key === 'id') {
return [key, val]
}
const highlightedVal = val?.replace(
new RegExp(keyword, 'gi'),
(match) => `<mark>${match}</mark>`
)
return [key, highlightedVal]
})
return Object.fromEntries(highlightedEntries)
})
return highlighted
},
},
watch: {
keyword: {
immediate: true,
handler() {
this.search()
},
},
},
beforeMount() {
this.loadData()
},
methods: {
async search() {
await this.$nextTick()
const { keyword } = this
if (keyword) {
const filteredSearchResults = await db.avatars
.filter((u) => {
const { address, city, email, name, title } = u
return (
address?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
city?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
email?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
name?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
title?.toLowerCase()?.includes(keyword?.toLowerCase())
)
})
.limit(10)
.toArray()
this.filteredSearchResults = filteredSearchResults
} else {
this.filteredSearchResults = await db.avatars
?.toCollection()
?.limit(10)
.toArray()
}
},
loadData() {
this.loading = true
const worker = this.$worker.createWorker()
worker.onmessage = () => {
this.search()
this.loading = false
}
worker.postMessage('load')
},
},
}
</script>
<style>
.avatar {
background: #bdbdbd;
width: 150px;
}
.right-pane {
background: #fafafa;
}
mark {
background: yellow;
}
#scrollable {
height: calc(100vh - 100px);
overflow-y: auto;
}
</style>
In the loadData method. we call this.$worker.createWorker to create the worker that we injected.
this.$worker is available since the inject-ww.js script is run when the app is loaded.
We set worker.onmessage to a function so that we recent messages from the worker when postMessage in the onmessage function of the worker is called.
Once we received a message from the web workwer, we call the this.search method to do the filtering according to the keyword value.
And we call worker.postMessage with anything so that the worker’s onmessage function in the worker will start running.
Now when we type in something into the text box, we see results displayed in the cards.
We should be able to load large amounts of data in the web worker without hanging the browser since it’s run in the background.
Conclusion
To run background tasks in a Nuxt app with web workers, we can add the Webpack’s worker-loader package into our Nuxt project.
Also, we’ve to make sure that web workers are only loaded on client side.
We’ll make a project that loads a large JSON file and searches it.