Updating Gdbgui with 0xide’s Frontend: Part 1
Motivation
While working on my OS, I’m constantly looking for tools to improve my development process. I was recently introduced to gdb’s tui (text user interface) mode by Greg Law’s Strangeloop talk. This mode replaces the arcane gdb terminal with a much more friendly user interface.
I wanted to see if anyone had improved gdb’s interface further. Surely the useful tool of gdb with the achilles heel of a horrible UI would have been fixed by now? Specifically I wanted something through a webpage, which would allow the UI to leverage nice front-end tools. I stumbled on gdbgui, which is an open-source project that does exactly that.
The tool is good, but the UI design is slightly outdated and clunky. At the same time, I was poking around 0xide’s frontend code and liked what I saw. So I decided to try and combine the two!
A bonus is that you could then run gdb in a UI-friendly way through Oxide’s console.
(Example of Oxide’s frontend. It’s a little dark and brooding for my tastes, but also much cleaner).
A quick first pass
I naively thought it would be a simple matter to update Gdbgui’s front-end. After all, you just chuck the new CSS file on the page, maybe update a few HTML elements, and you’re done! Right?…Right???
Turns out front-ends are more complicated than that. They’re not so complicated as to be unapproachable, but the many frameworks and layers of indirection can make browsing and updating the codebase unintuitive. For example, the HTML file only had a couple <div>s in it, while the React code renders the rest of the HTML. But where does React know which HTML files (or where in the files) to inject the generated HTML into?
So, after a couple days of grumpy-coding, I decided to take a step back and try and understand the frameworks on a high level. Hopefully by learning the project structure from the top down, the pieces will start clicking together. (I say grumpy-coding because that’s what it was. I was frustrated that I couldn’t understand something as seemingly simple as the front-end. I mention this because it’s a good reminder that beating yourself up while coding is rarely a good thing. I hope to continue to remind myself of that fact.)
Understanding Gdbgui’s frontend stack
Webpage rendering
My first question was: how does the webpage actually get rendered?
After digging around for a bit, here’s my understanding:
Browser:
+------------------+
| HTML: |
| <div gdbgui> <--|-- 1. Placeholder HTML is sent to used by the browser until React can generate and inject the necessary dynamic HTML
| ...loading |
| </div> |
| <scipt> |
| csrf_t={{a}} <-|-- 5. Where do these come from ??
| var = {{var}}<-|-'
| </script> |
+------------------+---------------------+
| JS: |
| <Data await fetch("/dashboard_data")> |<- 4. This React component will fetch data from the server via API and update the contents of the component
| |
| func Gdbgui() return <Data /> <-------|-- 3. The root React component that's used in render, has child element <Data>
| |
| ReactDOM.render( |
| <Gdbgui />, |
| document.getElementbyId("gdbgui") <-|-- 2. JS/React generates the dynamic HTML and injects it into a known <div> element,
| ) | in this case id=gdbgui
| |
+----------------------------------------+
(Hint: You can scroll left/right)
The browser has a basic HTML template with
- a
<div>with some placeholder text and a known id (in this casegdbgui) - a
<script>with some JS variables - a lot of JS created by our React code
As soon as the browser receives the page data, it starts running the JS code. React JS builds a DOM of the webpage and then replaces the placeholder text with known id with the React components. As our page is updated by user or server interaction, React will detect the change in the virtual DOM and update the webpage accordingly. From here, each React component is specified through an explicit import statement, so it’s easy to track the code-dependency flow.
Still, there are a couple questions that arise:
- How does the JS get the gdbgui data to display? (Presumably from
"/dashboard_data", but can we make that more concrete?) - Where are those weird JS variables coming from in the HTML
scripttags?
We’ll address these questions in order.
Updating webpage with data
First, understand that gdbgui, like pretty much all front-ends, gets data through querying APIs. Gdbgui does not live directly on top of a gdbgui process as one might expect, but actually has a whole backend server to manage connections between different gdb processes and Gdbgui consoles. This allows multiple consoles to connect to the same gdb process and vice versa, with the overhead of only one server.
The frontend can fetch data from the server via HTTP APIs and the server is responsible for updating the data and interacting with gdb. The server is written in Python with Flask and uses Blueprint for API routing.
How browser and server interact on a request for the homepage:
+---------------------+ +----------------+
GET "/" | App (Flask Server) | (connected | Kernel |
+---------+ --------> | +-----------------+ | somehow??) | +------------+ |
| Browser | (HTTP) | | Blueprint | | ,-----------> | | GDB pid=xx | |
+---------+ <-------- | +-----------------+ | | | +------------+ |
gdbgui.html | ^ | | | | |
main.js | | V | | | +------------+ |
| +-----------------+ | | | | FileSystem | |
| | Session manager | |-' | +------------+ |
| +-----------------+ | +----------------+
+---------------------+
Session manager maintains the state of each tracked gdb process and the server can retrieve data as required by API requests.
I haven’t found an easy way to generate an OpenAPI spec (a standard way of specifying an API) from Blueprint API annotations so I had to manually compile a list. Defining a more explicit contract between the frontend and backend would help decouple the implementations in the future.
Here’s a summary of the API the app provides with a brief summary of what each does:
Server HTTP API:
[GET]
"/" -> render_template("gdbgui.html")
"/dashboard" -> render_template("dashboard.html")
"/help" -> redirect to github's help file
"/read_file" -> Read file from file system and return file contents
"/get_last_modified_time_unix" -> Return the last modified time of file
"/dashboard_data" -> Return sessionmanager.get_dashboard_data()
[PUT]
"/kill_session" -> manager.remove_debug_session_by_pid()
[POST]
"/send_signal_to_pid" -> Kill process with signal
How Session Manager interacts with gdb
Initially, I was slightly confused when reading through the source code of gdbgui. It seemed like gdbgui is using two different API frameworks for communicating between the front-end and backend: some of the code uses @blueprint.route and some use @socketio.on.
Example Blueprint annotation:
@blueprint.route("/read_file", methods=["GET"])
@csrf_protect
def read_file():
Example SocketIo annotation:
@socketio.on("pty_interaction", namespace="/gdb_listener")
def pty_interaction(message):
However, once I separated the HTTP API out, I was left with only socketio annotations and only the code that deals with the terminals. Coincidence? I think not. It seems that Blueprint is being used for the server’s API requests and SocketIo for the gdb terminals. If you’re not familiar with Gdbgui, I’ve attached an image below. Basically, there are two main parts:
-
the top where all the gdb data, such as source files, threads, and expressions live
-
the bottom, which contains three terminals that are connected to gdb input, gdb output, and program output, respectively.
The terminal uses a separate connection, SocketIO, to connect the gdbgui terminals with gdb processes. My understanding is because this needs to be a long-living and frequently updated communication channel, a websocket is better than HTTP for transferring data.
After perusing the implementation, here’s a rough diagram of how the websocket works:
+--------+ +--------+
| Client | | Server |
+--------+ +--------+
| SocketIo.Event( |
| "Connect", |
| "gdbpid": 0, |
| "sid": xx, |
| ) |
| ---------------------------> | (newly created)
| | fork(), os.exec(gdb_command) +---------+
| | ----------------------------------------------> | gdb |
| | +---------+
| SocketIo.Event( | |
| "debug_connect", | |
| "started_new_proc": true, | |
| "pid: yy, | |
| ) | |
| <--------------------------- | |
| | |
| <If no websocket background |
| process exists yet> |
| | |
| | |
| | new SocketIo.background_process() |
| | +-------------------+ |
| | ---------> | Background thread | |
| | +-------------------+ |
| | | |
| | <Every 30 seconds> |
| | | |
| | | pygdb_mi.get_gdb_response() |
| | | ----------------------------> |
| | | <---------------------------- |
| | <If response> |
| | | |
| | SocketIo.Event( | |
| | "gdb_response", | |
| | response | |
| | ) | |
| <-------------------------------------------------- | |
| | | |
| | | |
(Hint: feel free to scroll around in the diagram above)
Note the websocket and HTTP API are completely independent ways of interacting between the client and server.
We can now update our diagram with our new knowledge:
+---------------------+ +----------------+
GET "/" | App (Flask Server) | | Kernel |
+-----------+ --------> | +-----------------+ | pygdb_mi | +------------+ |
| Browser | (HTTP) | | Blueprint | | ,-----------> | | GDB pid=xx | |
| | <-------- | +-----------------+ | | | +------------+ |
| | | ^ | | | | |
| | event | | V | | | +------------+ |
|+---------+| --------> | +-----------------+ | | os.exec | | FileSystem | |
||Terminals|| (Websocket) | | Session manager | |-' ----------> | +------------+ |
|+---------+| <-------- | +-----------------+ | +----------------+
+-----------+ event +---------------------+
And here’s a summary of the SocketIO events handled on the server end:
@app.before_request
def csrf_protect_all_post_and_cross_origin_requests():
@socketio.on("connect", namespace="/gdb_listener")
def client_connected():
@socketio.on("pty_interaction", namespace="/gdb_listener")
def pty_interaction(message):
@socketio.on("run_gdb_command", namespace="/gdb_listener")
def run_gdb_command(message: Dict[str, str]):
"""Write commands to gdbgui's gdb mi pty"""
@socketio.on("disconnect", namespace="/gdb_listener")
def client_disconnected():
"""do nothing if client disconnects"""
def read_and_forward_gdb_and_pty_output():
"""A task that runs on a different thread, and emits websocket messages
of gdb responses"""
Injecting variables into webpage
There’s still one open question: in our previous exploration, we noticed there were JavaScript variables being created in the webpage at the start in the <script> tag. For example, the webpage is served with a csrf_token that is checked in each subsequent request. But where are these variables coming from?? Well, now that we understand the end-to-end serving of Gdbgui webpages, we can simply walk backwards until we find where the data was created.
Looking in http_routes.py we find the following code (abbreviated for readability):
from flask import render_template
@blueprint.route("/", methods=["GET"])
@authenticate
def gdbgui():
initial_data = {
"csrf_token": session["csrf_token"],
"initial_binary_and_args": current_app.config["initial_binary_and_args"],
...
}
return render_template(
"gdbgui.html",
initial_data=initial_data,
...
)
It seems flask provides a standard way to pass flask/Python data into a HTML page during rendering before returning the HTML to the browser. Gdbgui also passes other default data to the webpage, which is later updated via API requests. For example, the default gdb run command can be updated by the user, which will appear updated on all subsequent webpage requests by default.
How project compilation/development works
So now we know how Gdbgui runs on a high level. But what actually happens when we run nox -s develop, which is the command in the README to build the project?
Nox compilation:
+-----------------+ pip-compile requirements.in +------------------+
| requirements.in | ----------------------------------> | requirements.txt |
+-----------------+ +------------------+
|
+------------+ |
nox -s develop ---> | noxfile.py | `---------------------------------------.
+------------+ |
Code: |
@nox.session(reuse_venv=True) |
def develop(session): V
session.install("-e", ".") ------------------------------------------> Installs list of requirements via PIP
session.run("yarn", "install", external=True) -----------------------> Run yarn install
print("Watching JavaScript file and Python files for changes")
with subprocess.Popen(["yarn", "start"]):
session.run("python", "-m", "gdbgui") ---------------------------> run Gdbgui with python
python -m gdbgui ---> Run module gdbgui
*This one I'm not 100% sure about, but my understanding is Python runs the module gdbgui, which is already built and managed by Python via the build command:
Code:
@nox.session(reuse_venv=True)
def build(session):
"""Build python distribution (sdist and wheels)"""
session.install(*publish_deps)
session.run("rm", "-rf", "dist", "build", external=True)
session.run("yarn", external=True)
session.run("yarn", "build", external=True)
session.run("python", "setup.py", "--quiet", "sdist", "bdist_wheel")
session.run("twine", "check", "dist/*")
for built_package in glob.glob("dist/*"):
# ensure we can install the built distributions
session.run("pip", "install", "--force-reinstall", built_package)
Understanding Oxide’s frontend stack
Now, it’s time to understand Oxide’s stack!
Webpage rendering
The entry point to the frontend is main.tsx in the app folder. Unlike Gdbgui, Oxide uses pure code instead of annotations to generate all HTTP routes:
// main.tsx:
// ----------------------
import { routes } from './routes'
const root = createRoot(document.getElementById('root')!) // <--- 1. Get the root element to inject generated HTML into
if (process.env.MSW) { // <-------------------------------------- 2. (Optionally start mock api then) Generate HTML through React
startMockAPI().then(render)
} else {
render()
}
function render() {
const router = createBrowserRouter(routes, {}) // <------------ 3. Create the HTTP routes
root.render(
// ...
<RouterProvider router={router} /> // <---------------------- 4. Render the routes
// ...
)
}
The HTTP routes themselves are defined in routes.tsx. I’ve included a sample from the file below:
// routes.tsx
// -----------------------
import { LoginPage } from './pages/LoginPage'
export const routes = createRoutesFromElements(
<Route element={<RootLayout />}>
<Route path="*" element={<NotFound />} /> // <------------------------------- If the user enters a path not defined below, fall into the default case of "not found"
<Route element={<LoginLayout />}>
<Route path="login/:silo/local" element={<LoginPage />} /> // <------------ Specifies the path (colon is used to get parameters from URL) and page to render
<Route path="login/:silo/saml/:provider" element={<LoginPageSaml />} />
</Route>
// ...
How the API is generated
Oxide.ts generates an TypeScript API from Nexus’ OpenAPI spec
oxide/console | oxide/omicron
+-----------------------+ |
| Typescript API |
+-----------------------+ |
^
| |
+-----------------------+ +-----------------------+
| Oxide.ts | <---- | Nexus.json | <-- Generated JSON file of control plane API from Omicron
+-----------------------+ | +-----------------------+
^
| |
+-----------------------+
| | Omicron | <--- Oxide rack control plane implementation
+-----------------------+
Here’s a snippet of Nexus.json, which contains the OpenAPI specification of Oxide’s control plane:
//...
"/v1/disks/{disk}": {
"get": {
"tags": [
"disks"
],
"summary": "Fetch a disk",
"operationId": "disk_view",
"parameters": [
{
"in": "path",
"name": "disk",
"description": "Name or ID of the disk",
"required": true,
"schema": {
"$ref": "#/components/schemas/NameOrId"
}
},
//...
Managing import statements
Oxide’s frontend keeps complexity to a minimum. This is intentional, as their README states, they want the client to be as lightweight as possible and not add any complexity on top of the APIs.
They also try to make their frontend code as readable and reusable as possible. As part of this, they make liberal use of barrel import files. Here’s an example of Oxide’s imports:
import { navToLogin, useApiMutation } from '@oxide/api'
import {
Button,
DirectionDownIcon,
DropdownMenu,
Info16Icon,
Profile16Icon,
buttonStyle,
} from '@oxide/ui'
versus Gdbgui’s imports:
import Breakpoints from "./Breakpoints";
import constants from "./constants";
import Expressions from "./Expressions";
import GdbMiOutput from "./GdbMiOutput";
import InferiorProgramInfo from "./InferiorProgramInfo";
import Locals from "./Locals";
import Memory from "./Memory";
import Registers from "./Registers";
...
Admittedly, it’s a somewhat cherry-picked example. There are certainly some less-exemplar Oxide imports as well. Still, the use of barrel files help bundle relevant code together. They also use aliasing in the tsconfig.json to import code as if were from a separate package. For example, the UI code exists in the same it repo, but the frontend still imports it as @oxide/ui. Here’s the alias in tsconfig.json:
"paths": {
//...
"@oxide/ui": ["libs/ui/index.ts"],
//...
}
I’ve already adopted a similar approach with Gdbgui’s imports and separated them into folders with barrel imports. It’s made the code much easier to reason about.
Headerless tables
Another way Oxide reduces complexity in their frontend is to use headerless tables. You can check out TanStack’s documentation here. Essentially,
Headless UI is a term for libraries and utilities that provide the logic, state, processing and API for UI elements and interactions, but do not provide markup, styles, or pre-built implementations.
It might be overkill for our project (after all, we do want to finish it) but worth keeping in mind.
Compiling the project
npm run dev
|
V
+---------------+ +----------------+
| npm config.js | --------------------------------> | vite.config.js | ,----> The three types of API endpoints:
+---------------+ +----------------+ | - msw: Mock Service Worker (fake API created by Omicron's OpenAPI docs)
Code: Code: | - nexus: Real nexus server connected with mock Omicron sled agent
"scripts": { const ApiMode = zod.enum( | - dogfood: Fully connected to real sled agent (requires VPC connection)
"dev": "API_MODE=msw vite", 'msw', 'dogfood', 'nexus']) --------'
}
"msw": { export default defineConfig(({ mode }) => ({...}))
"workerDirectory": "." ,------------------------------------------'
}, |
V
Explanation: `npm run dev` runs the build: {
"dev" script in config.js. Before executing, outDir: resolve(__dirname, 'dist'),
the node_modules/.bin is added to PATH. The rollupOptions: { input: { app: index.html }} -----> Used by webpack, which runs by Vite under the hood
script then executes the vite command }, ...
(from node_modules) with env variable plugins: {
API_MODE=msw. (Mock service worker) react: ({ babel: { plugins: [] }}) ---------------> Babel creates backwards-compatible JS code for legacy browsers
}, ...
server: {
port: 4000, ------------> Specifies the server port
}
Conclusion
Now that we understand the structure of the two codebases, we should have an easier time finding the right cut-set of dependencies to cut and combine into one project. Originally I had wanted to finish this project in one post, but since I’m running into some code issues, I’ve decided to split the article (and project) into multiple parts.
Stay tuned for Part 2!
…wait, Oxide also uses xterm?
Notes
-
One downside of Gdbgui is the server adds a lot of overhead to the gdb process. It would be useful to explore GDB stubs as a lightweight alternative for using gdb on a rack.
-
I created a number of diagrams during the process of understanding the code. They’re a useful reference but don’t fit in the natural flow of the blog post, so I’m throwing them down below.
On Connect when creating a new gdb instance:
Note:
- UserPTY is left terminal (GDB input)
- Gdbgui is middle terminal (GDB output)
- ProgramPTY is right terminal (program input/output)
+-----------------+
| Terminals | pty_interaction(pty_name="user_pty") +-----------+
| +-------------+ | ---------------------------------------> | |
| | userPTY | | <-------------------------------------- | Session |
| +-------------+ | user_pty_response | Manager | pygdb_mi +-------+
| +-------------+ | | | ---------> | gdb |
| | Unused | | | | <--------- | |
| +-------------+ | | | response +-------+
| +-------------+ | pty_interaction(pty_name="program_pty") | |
| | programPty | | ---------------------------------------> | |
| +-------------+ | <--------------------------------------- | |
+-----------------+ program_pty_response +-----------+
In depth for programPty
+-----------------+
| Terminals |
| +------------+ | GdbApi.getSocket().on("program_pty_response",
| | programPTY | | function(pty_response: string) => programPty.write(pty_response))
| | | | <------------------------------------------------------------------
| | | |
| | | | programPty.onKey((data, env) => {
| | | | GdbApi.getSocket().emit("pty_interaction", { data: {pty_name="program_pty"}})
| | | | ------------------------------------------------------------------------------>
| +------------+ |
+-----------------+
pty_for_gdb is for gdb running in kernel
Debug session:
+----------------+ +------------------+ +-------------------------------+
| pty_for_gdb | ---> | Pty(gdb_command) | ---> | if (gdb_command != null): |
|----------------| |------------------| | fork process and start gdb |
| pty_for_gdbgui | ---> | Pty() | | else: |
|----------------| |------------------| | connect to gdb process |
| pty_for_debug..| ---> | Pty(echo=false) | | |
|----------------| |------------------| | if (background_process=null): |
| pygdbmi_control| ---> | IoManager( | | start_background_process |
| | | pty_for_gdbgui)| +-------------------------------+
|----------------| |------------------|
| pid | ---> | pid of gdb proces|
+----------------+ +------------------+
Read and forward gdb
and pty output:
every 0.5 seconds:
+------------+ +------+
| | pygdbmi_controller | gdb |
| background | .get_gdb_response() | |
| thread | ---------------------> | |
| | response | |
| | <--------------------- +------+
socketio.emit( | |
response, | |
"/gdb_listner"| |
room=client_id| |
) | |
<-------------- +------------+
Websocket pygdbmi
Client <---------> server <-------> gdb
Compiling:
+------+
| HTML |
+------+
+
+----------+ +------+
| Tailwind | -----> | CSS |
+----------+ +------+
+
+------------+ Compile +-------+ Compile +------+
| Typescript | -------> | JSX | ------> | JS |
+------------+ +-------+ +------+