Welcome back everyone to Merch
Confiture Weekly Developer Podcast.

Talking about all things in the world of

software development,
AI, and new hardware.

I'm one of your co-hosts who works
at Microsoft James Montemagno,

and with me, the one and
only who used to work

at Microsoft, Frank
Riggas, and gone buddy.

Oh, it's been a while. Do they still call
it Microsoft? Is that still the name?

Is that trademark registered? Yeah. Okay.

MSO, still the ticker.

Tick tick.

Hi, James. It's wonderful
to be here on your show.

This is one of my favorite weeks where we
are going to talk about hardware spoilers.

I know we like to ease into our topics,

like talk about random
stuff for 15 minutes,

but I feel like we have a lot
to cover this week because

Microsoft's in the hardware game,
you know, like they put their toes in,

and then they step out of the water.
They're like, "Who's that?"

It's a little cold, and
then they get back in,

and then they're like,
"Eh, I'll try it again."

This is one of their, "We're just
going to cannonball into the water."

And I don't know what
you want to talk about.

What I want to talk about
is this big, cool laptop

they just released, and I just
want to dive into that puppy.

James, have you heard about this laptop
that a little MSFT company has released?

Well, a little company out
of Redmond, Washington.

I have one sitting in front
of me, so yes, yes, I know.

Okay. Well, aren't you rich because, man,

the starting price of
this thing is just as big

as its beautiful screen.

Well, no, there's a big hardware event,

but also a software
event and a local, like,

a model event too.
It was kind of... It was

like a windows and
hardware event, and to me,

it was also kind of also
somewhat a developer

event, a little bit too, I want to say.

I mean, we're getting all
the attention right now,

right? Because anytime
you say AI, it's like,

"Okay, business people
do something with it.

I'm really not sure what
business people do.

I assume they just chat with it all day or

something like that.
Maybe it fills out their

Excel sheets or something,
but I think you and I

have the most experience
with agents and everything

as developers, as I'm
sure everyone listening,

and we're kind of the crowd that all the

agency stuff is really
catered to at the moment.

So I feel like anytime
anyone's talking to AI,

there's a little bit of
our club talking, at least

for now. It's taken over
the world, for sure,

but we're the ones
deepest into this fiasco

called AI. That's whatever it's called.

Well, there was two devices.
One got the most

attention. The second
one is the one that I'm

also extremely excited
about, but I am excited

that you're excited for
Microsoft's hardware. I've

been a huge fan of the
Surface line of devices

for a long time.
They're beautiful hardware.

Frank, I'll let you lead, because I
know all the things about this device.

So got the sale you from
an outsider's perspective,

what I noticed, I noticed
Microsoft released

and partnered with NVIDIA
to create a new RTX

spark, whatever the heck
that is, Surface laptop

Ultra, the ultra being
very important because

there's a million surface
things out there,

laptop being important,
the ultra meaning it's fast.

This is an arm chip, and you will believe

how many web pages I had
to look through to find

out what the architecture
of the chip actually was.

This is an arm chip, everyone, and it's an

arm chip bolted to a
giant GPU, or it's a GPU

bolted to a small arm chip.
Not really sure which

one it is. All crammed
inside of a laptop case,

and speaking as an Apple Dweeb, this is

definitely a big competitor
to the M5 and the big

big powerful Pro Max that
have been coming out lately.

This is Microsoft stepping into the ring

NVIDIA joining them because
we need GPUs and they're

the only ones that can
afford RAM these days.

So they're all getting
in there releasing this

big beautiful monster of
a machine with a monster

price tag. I mean, not
forget to keep saying that,

but it's a beautiful machine.
Like you can't argue.

The video they used to
introduce it was very

Apple-esque, the machine
itself is very Apple-esque.

It's a big beautiful
Pixel Sense 200 and let's

just go with 50 pixels
per inch. It's high res.

That's all that really
matters. It's an HDR

monitor, and it's got
a ridiculous number of

NVIDIA cores in it. And as
someone who does a lot of

ML still, like I train my
own models and everything,

it's really cool to have
actual NVIDIA hardware on.

Basically, it's a personal
computer, I suppose.

It's more of a workstation
crammed into an laptop,

but it's cool to have that
NVIDIA hardware because

as much as I love my apples,
the metal ecosystem

just isn't nearly as
advanced as the CUDA NVIDIA

ecosystem. The kernels are
just a little bit different.

Some of them are buggy.
A lot of them are much

slower than the CUDA world.
Right now, my room is

20 degrees warmer than
it ought to be because I

have my RTX 3090 crushing
away on some numbers,

doing tensor math, using
all the flops in the world,

tops. I don't even know
what they are anymore.

That's all NVIDIA hardware.
That's CUDA. That's

my happy place. You can
run Python. Any model you

download from the internet
is going to happily run

on CUDA hardware very
efficiently, and that's

what they stuffed into
this laptop case. So for

$2,600, you can get a giant
GPU somehow attached to

a monitor and a keyboard
and a trackpad all glued

into a very fine-looking
case. It's a big price

tag, but gosh, it's a
beautiful machine. How can

you not be excited about it? You'd have to
totally hate tech to not be excited by it.

No, I mean, you're right.
It's a big, beautiful

machine. The starting price
is $2,700. It goes up

to almost $6,000. Now,
I do agree with you.

The real core is actually the 32 giga-ram

unified, so that's going
to get you basically,

I don't know, 3300 or whatever.
There's a bunch

of different tweaks on here.
But yes, yeah, I mean,

you're right. It's the
RTX Spark. I'm looking

over here so I can get
the tech specs. N1x,

it has either, there's two
models for the laptop.

One is 5,120 core GPU
with an 18 core CPU or a

6,144 core GPU and a 20
core CPU. There's a lot

of cores, Franker, and
it has, I don't know if

you've heard of this thing
called unified memory.

Can you just describe what
that means? Unified memory.

That's why it's an arm.
I should have known

just from the unified
memory that it was armed

because I don't know
Intel's still not doing it

for whatever reason.
Unified memory means the GPU

and the CPU are sharing
the same memory bank,

which is a wonderful thing,
A, as programmers,

because when I'm doing
development, I have to

first get my data into
system RAM, and then it has

to be uploaded to the GPU.
That's because I have

an Intel system with
non-unified memory. It's a

pain in the butt because
there are two different

speeds, honestly. DDR4,
usually as your CPU RAM,

some people have modern
computers, GDR6, 5, 6,

but it's usually quite a
speed difference between

the two, and in the past,
it was graphics RAM.

Graphics RAM was more expensive and faster

than stuff running inside
the GPUs. Nowadays,

it's getting a little bit
weird. You always had

this very awkward programming
model of you would

have to really write pretty sophisticated

software to keep feeding
data into your GPU,

to keep the GPU hot,
spinning so that your room

increases by 20 degrees
whenever you're in it.

That's very important.
But it's kind of a pain

in the butt because you're
constantly having to

synchronize these two, basically there's
two different computers in your computer,

and you're having to
synchronize the memory through

it as programmers.
They would try to hide it from

you, but like all bad
abstractions, it was a

leaky abstraction.
And if you wanted to get high

performance out of training
or inference, either

side of the ML world,
you had to get really good

at memory transfers.
You had to fully understand

all the DMA channels and
all that. Unified memory

obviously simplifies all that. I think it

premiered on the Xbox one,
Xbox, maybe just Xbox.

Apple's been doing it
forever. The beautiful

thing is we don't have
to do that synchronizing

visualization as developers.
The second beautiful

thing is when you increase
the RAM of the computer,

it's all just available
to the graphics card.

I love my RTX 3090, but
it only has 24 gigabytes

of RAM. And the machine has
16 gigabytes of CPU RAM.

So technically I have 40 gigabytes of RAM,

but can I use that for training?
No, James, I cannot.

I am stuck with 24.
And just two hours ago,

my training crashed because
I ran out of memory.

How app or po? Oh, wow.
Unified is great for two

things. A, you get more.
B, it simplifies the

programming model.
That's why we all like unified

memory. It's important. It's the future.

Some would say the past
if you're an Apple user.

Yeah. And I think that's one thing that
Apple's called out a lot in the M series,

the unified memory between
the CPU and GPU and

it's kind of the architecture
that is kind of like

moving forward enables a lot of scenarios.

And yeah, so this thing goes
anywhere from 24 gigs of

LPDDR 5X unified memory up to 128.

And it also goes up to,
it says a removable.

You know, that's the
dev box is a removable.

Is it is the surface laptop?

One removable. I think the laptop had
screws for the hard drive also. If I was,

I was, I couldn't tell
because I had a really

smooth video where the
whole machine explodes.

And I couldn't tell what
you're allowed to explode

yourself and what you're not.
But it looks like

you could unscrew that
hard drive from it says

removable says removal,
solid state drive, Gen

five SSD, the Gen four
for the 512, but Gen

five for the one terabyte
and two terabyte.

Yeah, the resolution of
the screen is 32 70 by

2180, 262 PPI on it.
And three by two ratio,

like all the other service devices,
H.R. all that stuff that's on there.

But I think in general,
yeah, it's a, it's a,

as you saw, like a super gorgeous device.

And it's a really high
end device. I think what,

Pavan was saying was you
need device for creators

and the kind of like Harkins to what Apple

originally had like with the pro model of
devices. And now they saw the, obviously,

the pro and the studio
models, but then they

delineate them with the
specific like chips,

like your pros and your
max chips that are out

there. And, you know,
by the name of this, the

ultra, you know, that
is more than what the

surface laptop and the
studio would be is like,

they're saying, this is
the ultra, this is the top

top to the top end that
you can get for creators.

And they showed a lot
of different scenarios,

specifically of running
a lot of local models.

And we'll talk about
local models and training

and things, but a lot
of different developer

workflows, creator workflows.
There was one

specifically that they
showed about an artist,

that does a lot of like different artwork,
you know, kind of like mock up artwork of

space and stuff that you and I, you know,

gel with those kind of
very like Firefly ask.

Serenity asks, but like
that type of that world

is kind of like, you know, illustrations.

And he was running it through all of these
different models that once he had drawn,

drew, draw the base of
it, then he could run

it through all these models, like locally,

like five different
local models, some cloud

models, do a whole do a
whole screen, go through

blender and do all this work all locally.

And he was like, things
that used to take me like

weeks now take me like
a day to do like all on

this device, like with
the power of it. And

it's not just these models,
but also the, they're

talking about super nemotron.
And then also the

MAI code one flash model,
which is 120 gig, 120

billion perimeter model
running locally on these

devices. And I think that's
like super crazy impressive.

I'm still running, having R. Dem running

any models on my, you know,
M4 Pro 64 GBS. I can do it.

It's just a struggle.
That's all I'm saying.

It's going to be a struggle.
I mean, except for MAI,

it's going to be a
struggle with everything.

Running a local model
sadly is a little more

complicated than it needs
to be. But you hit the

nail on the head because
you can make the most

beautiful machine in the
world, but you need to

have good software to run it. And that's
kind of what I was saying about the, the,

CUDA and video world.
That's the one nice thing

about that world is there's
a lot of AI software

out there tuned to it,
but it's not just that.

It's video production software
is tuned to it. And video

not only is a large
profitable company, but it

turns out they've taken
over a lot of the software

ecosystem. So a lot of
your high end plugins for

things take advantage of
all these Nvidia chips.

We shouldn't forget gaming.
This is going to be

a monster gaming device,
but I don't know what

gamers are going to spend
the way they showed gears

running on it while they
were doing all this stuff.

Yeah, definitely more
for the creator crowd.

If you're doing like a lot
of CAD, if you're doing

movie production, all that
stuff, things that can

take advantage of these
ridiculous GPUs they're

putting in there.
But yeah, building up that

software ecosystem is difficult.
So being within

video, they're lucky.
They're just inheriting a

bunch of software that's
just going to work with it.

But then they made a
bunch of enhancements to

Windows also that's going
to take advantage of it.

The one that I found
kind of interesting was,

and I'm you're going to
have to help me here,

because I could never
quite remember its name.

But there's machine learning
on Windows. It's kind

of the equivalent of core
ML on the Apple side

where you can kind of
recompile your models to run

nicely on Windows.
And I believe like, on X models,

ONNX models, which is
an interchange format,

has like native drivers
to run officially on

Windows. So it seemed
like a big part of this

announcement wasn't just
this hardware, but all

these software upgrades
to Windows specifically

for being able to run these
more advanced scenarios

more easily. Because you
could always do it like

I'm saying, you could always
get torch up in Python

and get these things cranking.
But it wasn't easy.

It wasn't fun. And so I
think, and you've had that

experience on Mac.
So getting these things to be

easy to run is important.
Yeah, and that's what it

was specifically.
So there's a big blog post on

the Windows experience
blog, which is, is late

to power this. So Windows ML is an AMVIT.

Thank you.

It is a scalable runtime
to deploy model across

GPU MP and CPU. And then
they announced support

for llama CPP, giving developers more
open source choices and also, pie torch.

As well for Windows ML.
So pie torch and also

llama CPP, bringing
those directly into it,

which is pretty rad.
So in general, what that

looks like, they have a
diagram on this website is

like your native app or
web app. There's like a

custom open source model
for on X, pie torch or

GGUF goes into Windows ML
and then interlaces with

the Windows Silicon
environment, whether it's

Intel, Qualcomm, AMD or NVIDIA. And then

that accesses the GPU,
MPU and the CPU too.

Yeah. And let me kind of
break this down as like

a practitioner of ML.
You're going to use your

pie torches and NVIDIA
CUDA stuff to develop

ML stuff. And it doesn't
even have to be large

language models. It can
be all sorts of things.

It can be the imaging stuff.
It can be audio stuff.

It can be video stuff. But that's going
to be like your development environment.

llama C++ is very nice
for the nerds out there

who are building their own harnesses and

development environments
and everything that

are going to be executing
these models. Because

a lot of the open source models out there
come in llama C++ native formats GGUF.

And so you're just going to need that file

format and something that
can execute that file

format. That's why it's a
big deal that llama C++

comes out. But as app
developers, if you actually

want to deploy the stuff,
you really honestly don't

want to use either of those.
Because deploying

torch and Python is gross
and annoying. llama C++

is just hard to keep up with.
It's a moving target.

So that's where the nice
Windows ML. It's nice

to have a stable deployment
target. And in the

past, it was onyx.
But onyx has onyx is kind of

just the file format.
So there can be different

engines that execute those onyx models.

So as developers, if you want
to add custom ML stuff to

your software, you want
to be deploying with

something stable and platform
approved. And that's

where like Windows ML comes
into play. So it's all

those different areas like
from development up to

nerdy stuff or you're
willing to get your hands

a little dirty up to this
is the clean polished

rosy path for deploying
your things. And it says

here, you know, they're
contributing back to llama

CPP and in partnership
with NVIDIA to your point

here in the broader community.
They're delivering

a bunch of performance
improvements to llama

CPP and GUF support in
the format through CUDA

kernel optimization,
kernel fusion improves

CPU, GPU scheduling, weight repacking, and

kutigrafts that work also
adds Eagle three, MTP

and D flash two, speculative
decoding, multi-GPU

execution, NVF P4, new
model architectures and

back end samplings, all
sorts of stuff. I don't

know what any of that
means, but sounds cool.

I could tell you every
single one of those, but all

you need to know is that's
the modern stuff invented

to make these things fast.
We're just trying to

go fast. That's all we're
trying to do. It's easy

to train these things,
but making them fast. But

it's funny, that's not
actually how the presentation

started. They didn't
start with fast. No, what

did they start with?
Do you remember? Well,

we started mostly with I
think I think Kayla was the

first demo. Was she the
first demo or the second

demo? Even earlier,
I think it was safety and

security James. It was
actually, yeah. I think

Satya said safety was
actually the very first thing

that he mentioned.
I'm not really with him.

I say throw caution to the
wind, let these things

have full access to your
computer, give it every

login you have, allow it
to use all the memory in

your house. Just put RAM
inside the computer and

turn the crank until it
has enough RAM. That's

what I think. But who am
I to argue with Satya?

What they announced was
the Windows container

support. I don't know, was
it in a beta or something?

Because they made a big
deal. It sounded like

this was almost an official
release or something.

Was that what happened?
It was in preview a build.

We announced the Microsoft
execution containers MXC.

MXC specifically works
across Windows, Mac,

and Linux. What it does
is it gives process and

session isolation.
Obviously, on the different

operating systems, it's
going to use different

container systems that
are there. Obviously,

on Windows, because it's
the stack that they own,

they can go a lot deeper
on it. On Windows,

it has a lot deeper process
and session isolation

through MXC locally with
these one-off process

isolated sessions.
It now supports WSLC, virtual

machines, and Windows
365 for agents, which is

really cool. I was demoing
this myself the day

before. I'm very in
tune with it. Actually,

by demoing it, I'm in love
with this more than ever.

It was cool that it was the first thing.

I had been -- this is
just sandboxing people.

We heard about sandboxing
before in VS code and

the Co-Pilot app and the CLI. MXC now is

rolled out across codex,
Co-Pilot, OpenClaw,

Replit, LM Studio,
Unscent AI, and a bunch more

that are releasing support
now that this thing

is GA, which is super big
because the standardized

support gives us developers.
It's actually just

an open source library that
you can integrate into

your application to provide
this functionality,

like everyone else has.
Then as users, we get

this really rich features
and capabilities.

So you can think of it as
you're able to control

disk access, network access,
clipboard access,

all these other things that you're able
to really get fine green control on.

And what the first thing
I did was I'm like,

why ever what I want to give, for example,

the CLI, VS code, or the app access to my

OneDrive, my work OneDrive.
So you can just

say block this folder.
Just like, you never need

to go to my OneDrive.
That's my business stuff,

right? So you're there.
Now, a lot of these

sandboxing are really nice.
So it'll be like,

hey, I'm trying to access
this file. It's in a

sandbox. It's a shik did.
Am I allowed to? And

you could break out of the sandbox. If you

need to, there's always
those gates just like,

do you approve this thing
or not? Now auto approval

on this thing, it needs
this human interaction

to do that. And it would
like turn it off with

that call and then turn
it back on and back and

forth, which is fascinating.
But that's up to the

client, that's up to the
app developer to enable

that functionality. Now,
the real cool part about

it is that most of the apps
today are doing process

isolation. So that means
for most intense purposes,

and that can be, that could
be like a five second

boost up to like they could
have a process that's

running that's isolated.
So when you boot up

codecs, like every session
has its own process that

is then isolated, but
you could make it so you

make a request an agent
and as soon as that that

turn is done, you kill it,
right? You could do

that as an option.
Normally, even with the CLI

or VS code has a process
that's running that's

sitting there for each of the sessions.

And those are in isolated
process there. However,

the thing that's even
the cooler than that is

session isolation.
And session isolation is cool.

And because Open Claw
has it integrated today.

And what session isolation
does is it actually

gives a full like user isolation for that
application running. So the Open Claw

that is running, the
agent that is running in

Open Claw has its own
basically user account

and its own user folder
that you can peek into,

but it can't poke out
to, which means every

process that is spun up
in the session of the

app, like the entire app
is running as its own

separate user account,
which is 100% then isolated

from you. And that's like
built into this thing.

And that is like super
crazy cool, actually.

I thought that was like
really, really neat.

And I've just like really
fallen in love with

it, funnily enough,
because I was demoing it

nonstop and I was like,
oh, crap, this is like

really cool, you know,
in general, you know,

and then you can have
like a allow list and

denial list and read only,
write only the only

things. The only problem
is you really got to

set it up. You know,
that's the main problem

in general. However,
what I'm really thinking

is, okay, let me have
some standard defaults.

Such as maybe don't be
able to do this stuff in

my one drive folder,
your drop box or whatever,

you know what I mean?
So just some like default

standards, but you could
imagine in the enterprise,

this is like crazy important.
And they showed a

bunch of other windows
features of like really

deeply integrating these
things were really agents

almost really have their
own identities on their

on your machine, which is
like so cool. Which is

right, you know, Unix
invented this in the 70s.

Yes, user accounts are
good and they can be used

for other things are than
users. That's just a

good isolation mechanism.
So it's good to see it

actually extending.
Honestly, I have a little bit

of mixed feelings here
because I think with any

new security model, there
is that time to figure

out what should be the
defaults and whatnot.

You know, so every time I
open a session in VS code,

it has to ask me permission
to like read something

from Microsoft. I'm like,
you can you can use

the Microsoft.com website. It's fine.

It feels like no matter how
many times I say allow,

a new security model
gets introduced and that

allow is turned off or
something like that. So,

as a user, I get a little
bit frustrated with

there's a little bit of
theater with our security

and there's a little bit
of real security with

our security.
And I understand those differences,

but it can still be
frustrating to find that

fine line between them. And I just totally

understand from the
enterprise standpoint why

they want it. But some of
that stuff coming back

onto real users, it's a
bit frustrating. I mean,

talking about Windows
Vista, like that we were

the first ones to add the
annoying pop up dialogues.

Hey, do you really want
to do this? Do you want

to give permission to do
this? And it took quite

a few versions to get
those dialogues refined.

And I feel like we're still
just in that annoying

stage. But hopefully
with MCX coming out and

everyone kind of consolidating
on it, we'll have

some a better defaults and
be just, I can just say,

hey, yeah, here's a whole
set of directories on

the computer. Go have fun.
I don't care what you

do over there. Exactly.
And I do think that this

is going to be ever important
as these agents are

sort of like autonomous,
they're just running on

machines separately.
Like you said, you know,

it's like, yeah, this
thing is just running over

there. And it's I'm not
even like, I'm not even

know what's going on, right?
And you really want it

to be its own separated,
isolated process. Yeah,

MCX stuff was really cool.
I really like, I really

dug it. I was super
excited about it. And just

how easy it was. And now
that I've dog deep friends

actually using it a bunch,
it's definitely worth

checking out. And there's
like definitely some

smart defaults. I think
the problem is like if you

have sandboxing and then
you have like, there's

a few things, right?
So for example, there's,

there's like sandboxing
to say, Hey, listen,

I want you to run all my
terminal commands in a

sandbox. That is one thing
versus a network, disk

access, things like that.
Because the problem we

run into as app developers
is, Hey, I need to do

a donnet build. Oh,
donnet's not installed in the

sandbox. Yeah, right.
That's a problem. Whereas

what I'm talking about
is more permission based

sandboxing for accessing
locally on your machine.

So I do think when we
think sandbox is a pretty

loaded term. But what
I was at least setting

up as my default for MXC. As far as like,

file and network access,
which networks access

for me is pretty much like,
yeah, it's fine. But

you could imagine an
enterprise, you could just

say, I only want to access
like local things. I

don't want you to go out
to the internet that

would probably like import
for them. But there's

this and one of the cool
parts is like, it also like

proxies the access through
that. So for example,

I had this demo that was,
Hey, can you go get

me a list of like my
GitHub issues or whatever.

And it was going to use the
GitHub CLI to go do it.

But the GitHub CLI is
running inside of the

sandbox with the sandbox
networking, which then

blocked it because it can't access GitHub.

Because I turned off network
access outside of it

on this project. So kind
of interesting in that

regard. So I think it's
just fine line of like,

okay, what am I, what things
am I doing versus like,

I just want to build a
project, like just run it

on my machine because I
have all the stuff on my

machine and not in the
sandbox. And that's what

kind of is right thing
versus agent versus like,

developer build tools, like those
two worlds seem to kind of like.

Right. But I feel like
we're the first beta

testers of all this agent stuff. Yeah.

All the under other industries,
they're still in

the chat world. You know,
they're still doing

everything at the chat
prompt level. We're the

ones doing the agent and
craziness, let it take

over my computer stuff. And that line,

it still is a shaky line.
Like just yesterday,

it couldn't build my app. And I'm like,

why can't you build that app?
They're like,

dotnet restore is stalling. I'm like,
why would dotnet restore? So they're like,

because I don't have
network access. I'm like,

oh, okay, why did it take
a why to take me three

wise to find out you don't
have network access?

And B, I'm like, why don't
you just enable network

access? And it's like,
great, I will. And then the

Mac dialogue pops up saying,
Hey, do you want to

give it network access?
Mac OS 27 changed that

dialogue box. So it took
me a while to figure out

how that dialogue box
works. Good. Finally,

I granted access.
And it's like, Hey, you have to

restart the harness in
order to do this. And I'm

like, but I'm in the middle
of a session. So I'm

just deep breath. That's
like the security stuff

is a little bit.
So let's just hold them down.

So that's good. We'll work it out.

We'll figure this stuff out.
How hard can it be?

Well, the next thing that
they talked about beyond

that, right? And also,
like at the very end,

we'll talk about the other
hardware that I'm more

excited about. So there's
the another piece of

hardware. But we'll
talk about every single

announcement here as we
get into like some deeper

Windows stuff. But one
of the things that they

really focus on was sort
of like local and hybrid

AI model usage. And one
of the things that Kayla

showed off, because on my team, which also

worked on the Windows
terminal team, just to be on

the Windows terminal team,
is basically frontier

coding models, like
local and your machine.

So there's a few things
is like on this machine,

what they were talking
about is like, Hey, taking

me I code one dot one flash
with Nematron models,

also deep seek V4 flash,
like that is a 137 billion

per parameter. You know,
total 6.8 billion active

and video is a 70 billion parameter, two

bit quantization down
to 20 gigs on memory.

Deep seek V4 flash is a 284
billion parameter model.

So there's all these
things that are going to

run locally on the RTX
spark, which is crazy.

Obviously, I'd imagine,
depending on what RAM you

have, you want all the RAM,
by the way. And they're

optimized and tunes to
basically run on these

machines. Now, of course,
you can run those

Lego machines, but you
also need great tooling

to run things locally.
And this is where the

brand new local model
integration in GitHub

co-pilot comes in. And there's two things

specifically. One is that
Kayla showed off like

auto local routing and
specifically that was

turning into efficiency
mode and the model.

The router can route to
a local model on disk

automatically. And this
efficiency mode has a

little like, a little leaf
icon, which is very cute.

But the thing that I
think is even more cool

is the hydrophusion support,
which you did an

entire podcast on.
And hydrophusion can then

decide to delegate out different tasks to

different local models on
disk and make decisions

optimizing your token
usage across the board.

And this thing is coming to
like everything, basically,

out of the box, along
with all that sandboxing

things that happen.
And Kayla had a really good

demo, which was, hey,
like, I've done this repo,

and then I'm doing these
investigations. And I

might be using like a
frontier model in the cloud.

And then it may decide to
then delegate locally,

you know, accordingly for that, those
different tasks can run off subsections.

She also showed off,
I think she also showed

off automations.
And actually, this is actually

really genius. And I was
talking to Kayla about

her usage of local models
on her machine. And she

said a lot of what she's
doing is, yeah, she

does have, if she's like,
hey, I'm about to do

something that is going to take a while.

And I don't actually care
about, like, I mean, time to

first token is important,
like, you know, and then

token, like, coming back.
But if she's like, hey,

listen, I know that my local
models are never going

to run as fast currently
as like, if I'm running

like Luna to go do a thing.
However, if I'm like,

hey, I'm about to jump in
a call and have a half

an hour and an hour or
whatever it is, or overnight

to run long processes,
what do I care if it's

running locally on my machine
or if it's running in

the cloud, right?
Besides that I'm going to save

a bunch of money.
And then she said, she actually

has a bunch of automation
set up that run every

single night at midnight when her computer

just sitting there with
nothing else to do,

and is then crunching
the local models to do

a bunch of issue triage
bug triage support,

like kind of like these
workflows automatically

that can then fire off
new sessions or do a

bunch of computation. And she's like, now

that's all just free for
me, right? And then,

she also showed, you know,
like what she was showing

in the hydrophusion examples,
it may start with

a cloud model, local model,
go back to clicking,

go back and forth, but
it's going to figure out

what it needs to do and
what local models you have

readily available, which
is like super crazy to

think about. So as she I
don't know if she showed

it, but we were really
showing is like, you now

could basically for all
intents and purposes,

like just boot up something
like this, turn on

airplane mode and go, right?
So yeah, it's so cool.

Your computer will get
warm during that process,

though, I promise.
So you can use it as a heater.

You know what it is,
it's it's liberating. Maybe

you don't experience
this because I think you

have infinite tokens.
But for those of us with

limited tokens, I'm always
worried about like,

I'm always watching the
like, how much usage do I

have left? Am I over my
budget for, you know, I,

I'm always like looking
at the date, oh, it's the

10th, there's 30 days in
the month, so I should be

at 30% and you know,
I'm doing this like stupid

math in my head. But when
you use a local model,

and even a stupid one,
it's very liberating because

you're just like, oh,
I have as much as I want,

I can just have this thing
chugging and burning

whenever I want. There's
there's no little background

clock in the back of
my head. Yeah, I mean,

hey, don't do this task
because you want to save

your tokens for that other
task or things like that.

When you have a local
model, you're just like,

I'm gonna use it and I'm
gonna, yeah, even the best

models out there, I was
just yelling at Astra and

Astra is using up my
entire month of credit in

like two hours when I
use it fully. And even

then I'm yelling at it
because I'm like, yeah,

that's an ugly button,
you made your button ugly,

it should be better than
that. And I find I yell

at my dumb models and my
local models just as much

as I yell at Astra.
So I would rather be burning

just electricity here
in my home and yelling

at the model than worrying
about token counts

that I'm paying for cloud
models and all that.

So if you haven't been
lucky enough to experience

local models, I don't
want to go and say like,

they're just as good as
the big cloud ones because

they're not, they definitely
aren't. But what they

are is liberating.
You just have zero guilt. You're

just like, I'm just
going to use this model.

Maybe it's on an airplane.
I don't fly that much anymore.

So it's fine. I don't
care about the airplane

scenario. But what I
care about is just not

worrying, not thinking about my
quotas and any of that stuff.

We're sitting there doing
these demonstrations and

when a demo or was was
booting up one of these

models and just like
let it rip for like 30

minutes and after 30 minutes,
I had like a full

video editor. And I was just
like, cool, here it is.

And there was this whole
thing was an airplane.

But I was like, really
kind of crazy. And like

you said, very liberating
and like, you know,

token used to zero. I think you need to,

it's a blend, right?
You need to use the right

model at the right time.
And then, you know,

especially with those like
long running processes,

like I had a session
that ran for 100 hours

fully, you know, I was
cranking and testing some

things out and just go
into town, right? And this

is like a blend between
frontier models, local

models, all these other
things. I'm bopping back

and forth between them.
And I think like that's

a really unique scenario
where I was just like

set in, forget it.
You know, I'm going to be in

San Francisco, just like,
I'm not even, I'm not

even having to have this
computer, my computer is

at home. What am I doing?
I'm not even checking

in on it. I was like
autopilot, Yolo, I'll come

back on it. It's sandbox.
It's good to go. And I

think that's like a very
intriguing scenario.

I mean, the other thing
that they also announced,

but actually didn't like
show, I mean, they showed,

but they actually have these like AI super

computer, like tower things
like that. I don't know.

I don't know if it's, I don't know if it's

not to call them like a
token factory or something.

I forgot what he said,
something like that.

Well, basically there's
these huge like basically

servers that, you know,
kind of server towers,

you know, DGX stations,
which are these huge

stations that they're
talking about running

windows now. And these
are apparently using a GB

300 grace Blackwell ultra
chip DGX workstation.

It says it can unlock power to run
frontier models from Loma for Maverick,

Kimmy, K to six deep seek
for V for pro and more

than a trillion parameters
on these things.

Oh my god. That is a great
are the professional

grade model runners.
Maybe someday I'll get a

10 year old one that
would be nice. I think

there was a cute little
afterwards during the

presentation where they
were just reminiscing about

like the computer industry.
And they're like,

do you remember when chat
GPT was trained on a

DGX one? Do you remember
that DGX one? I'm like,

guys, that was like three
years ago. It's not

that long ago. Like,
I would still kill for that.

And they're acting like
it's all old hardware.

They're like, Oh, we just
have those. We put those

out the pasture, shot them
and left them for the

carry in. Like guys,
I'll take your old DGX.

Just Satya, Frank will take
your old DGX. It's cool.

It's cool. Yeah. Well, there's a few other

things in like windows, windows itself.

So there's a lot of things
around performance,

search optimization,
making search like the most

amazing thing in the
entire world. I think they

really focus a lot on here
the improvements that

were making to core windows,
how we're making a

sort of an agent first operating system.

And I thought that was really
cool how like agents are

deeply integrated.
And also I think how they're

evolving co-pilot.
And specifically around this,

whole idea that you
have this home code and

autopilot and home is
really like, I am on my

machine. I have the context
that is my in this case,

like I'm in now consumer
mode, right? Like,

I want to write documents.
I want to do stuff.

And I want to be safe.
And I want to not just be,

you know, you know,
yellow on it out to wherever

context, but basically you can, you know,

co-pilot can understand
relevant content on the PC,

including files, recent
activity, local context.

And the cool part about
accessing these is it

can also tap into local
actions. So it can take

actions on your behalf. So it can organize

files, diagnose device,
troubleshoot issues,

it can obviously code,
it can carry out workloads,

it basically do anything
you want. And I'm thinking

like this kind of like,
what's the next general

of like shortcut type of
thing, actions. But then

it all goes so can tap
into local models too,

running on your device.
So when it's accessing

these things, it's able
to access the same models

that you might be using
for coding or these other

things actually do this
work, which is really

cool. So they're really talking about this

hybrid intelligent MXC support built in.

And then the idea that you're
going to have these raw

long running personal
agents too. And to your

point of like, what are
people doing? Like,

actually, the thing that
I'm doing at work often is,

you know, inside of co-pilot,
like a Microsoft

co-pilot is I am for all
intents and purposes,

having it gather a bunch
of my work intelligence.

Like I do this is a thing
that I do. Like I'm

planning right now a
bunch of our co-pilot,

a get up co-pilot like
advocacy efforts, right?

What are we doing for the
rest of H1 and in H2?

And what I needed to do
is I needed to tap into

my emails, to my messages,
into my files on disk,

and into these conversations.
And I needed to tap

into other planning docs,
coerce things together,

look at latest news on the
internet, pull things in,

and then take this and
then transform it into

docs and reviews and these
other things, and then

messages and send things
and do follow-ups and

then that's the thing that
I'm actually doing at

work, which I wasn't really
doing all that much,

like six to 10 months ago.
But I've been trying to

do a lot more of those
processes as these tools

gotten better.
And I actually find myself doing

this too often is in the
co-pilot app, what I do

end up doing today is I'm doing a lot of

house projects, right?
And I'm doing a lot of

contractor reviews and
all these other things.

And right now I'm dragging
and dropping files

into a thing and doing like,
I would love to just

be able to do a lot of
that integration into

Excel and into these other
things in this workspace.

So I'm really excited
about how that's evolving

and how these, you know,
agent harnesses are

running. I'm not sure
if I'm there yet. Like,

I'm not an open claw Hermes
person at this point.

Like, I use them in some
regard, but I'm starting to,

I think once you start
to see how other people

work that I'm interested,
like, I'm not really

interested in giving meta all my data.

Like everyone's like really
excited for a muse. I have zero

interest in giving meta
all of my data, right?

Like, I'm not interested
in that at all. I am

really interested in local
models being able to

do computation over myself,
which I think of

Apple intelligence, right?
Those models that are

running local and your
device like, you know,

Apple, when the local
model trend is there and

you know, they've been
trying to evolve that in

the latest devices and
the leading operating

systems. And I do like
this idea of hybrid. I

think this is, I don't
know if I don't, I've

been tried Apple intelligence
and in Mac OS 27

yet on my machine, which
I have here. But from

my experience of what I
saw when I saw people

trying to use Siri was it
would ask you to question

and say, Oh, I can't do
this right here. I'm going

to go to the cloud.
That's almost hybrid, right?

Except for you're doing
these actions. In this

case, it's automatic hybrid.
It's going able

to be able to do these
smart things locally,

remotely and configure
this for you. Now, I think

that the biggest thing
that Apple got right,

then I'll be interested
in this experience is

the seamless one button click, turn it on.

Like, I think that's really important.

Like inside of Xcode,
inside of your Mac OS machine,

inside of your iPhone, you say, do I want
to use Apple intelligence and you say,

yes. And then that's all
you don't have to worry

about what model am I using?
What am I doing?

XYZ? That's what I would like to see from
this experience is like, if you want to go

bits and bop and pick
your models, go for it.

But we're going to give
you a bunch of defined

models that are just really
good at doing certain

things that you're going
to be doing locally

anyways, like doing text or doing image
stuff or doing like profile processing.

Yeah, absolutely.
And you're really hitting the

nail on the head.
And I got to say, I don't think

either operating system is quite there yet

because ideally in the
future, in the same way,

I can pick which search
engine I want to use in my

browser, I can just say
use Google or whatever,

use Bing. I would like to
be able to under Syrian

Apple intelligence or
Syria or under Windows ML

just say like, here's a
model I downloaded from

hugging face use that for
everything. That's the

model I want you to use.
I want you to use it locally.

Don't log it to anything.
This is the model.

But sadly, right now
we're still in the case

where Apple releases one
model. It's the model

that comes with the
operating system. The model

that comes with Mac OS 27
iOS 27 is different than

the model that came with 26.
And I feel like

Windows is going to kind
of be there right now too.

Plus Windows is in a really
good position because

I got to tell you Apple's
dropped the ball a bit

still with Siri.
I've been using it for a while.

I am not amazed.
I was arguing with it all afternoon

where I would keep asking a questions.

And I wish it would say the
thing like you said of,

it'll say, hey, I'm going
to go use Chetchi PT or

whatever right now. No,
all I kept saying was,

I didn't quite catch that.
Do you want to try again?

Like, no, you obviously
got the text. I can see

it. You printed the text
right there. You got it.

You're just not acting
on it. So for whatever

reason. And it doesn't make
any sense to me because

I know I use the local model.
It's like it can

do this stuff. So I don't
know. There's there's

Apple's dropping the ball.
I think this is a perfect

opportunity for I'm in
Windows to catch up but

Windows to get ahead quite
a bit because the demo

they had for the start
button and search like

they were saying it's
faster than you can type.

Now I know some fast type is out there.

So I'm curious how just how
fast it is but like the

demo looks great. I know
it's easy to make a demo

look good but you know
time will tell when people

get this installed on
their machines but I think

they have like Apple should
have the lead in the

same way Google should
have the lead in AI.

All these people who should
have the lead don't.

So this is a great
opportunity for Microsoft to

like really make some headroads.
And I know they've

gotten a lot of flak
like co-pilot this and

that and the brand is getting a little bit

overwhelmed and trying
to put AI and everything

but like the Windows search
box is a perfect place

to impress people.
If you start impressing people

on that box you're going
to get a lot of converts

and like they're just
going to get used to it.

You're going to set a new
bar, a new standard for

what your computer should
be able to do and help

someday Mac will catch up
to them at least to that

demo. Well and I think if
the users are starting

to get used to this aspect
like when you Google or

you Bing or you like
sort something into some

box and you start to get
these AI summaries and

you start to follow up with
AI you start to get used

to this and you almost
want that as the default

experience. So it's almost
like looking forward

okay or people are people
there now right. That's

the problem is you try
too early or your first

one of the first there
you know it's going to

take a lot of revs to
get it right and you're

going to maybe make a
lot of people mad and

you're going to make some
people happy and then

you know maybe you all
mock it and hopefully so.

But let's talk about
the last thing. There's

plenty more things that we could cover but

a thing that I am most excited about in

this moment is not a laptop.
Frankly. Okay.

It is a PlayStation 2.
No it is a brand new it is

a brand new it's a brand
it kind of looks like

a hybrid between like an
Xbox and a PlayStation

2 like slim which is the
Surface RTX Spark Dev

box and I'm pretty sure
they showed this app build.

I think I'm pretty sure
it was there and if not

well something was there
but okay right I think it

was there but this for
all intents and purposes

is very similar to the
ultra however there's one

configuration you get one
you get one that's it

one price one configuration
and this really

is I believe it has a
default out of the box

Windows 11 experience for
developers like it is

it is a special image that
is made for developers

I don't know what that means but they said

it on there from like
what it's supposed to

I'm pretty sure it's
what it is but it is a

the same 6144 core GPU
20 core CPU the biggest

it is 128 gigs of LPDDR5X
Unified Memory and it

is the biggest SSD the
two terabyte which is

removable by the way in
this puppy and this thing

looks just gorgeous it is
a big ish not super big

but big ask basically dust
on PC and it starts at

6000 lives yeah 559 yeah
add us tax to that you

owe the dollars I've
held it it's beautiful

it's fantastic and yeah
it's it's what I want

I think like I know it
doesn't have a screen

Frank but also our Mac mini
zone have a screen and

I I tweeted after the
event that it is official

that my Mac mini is
officially for sale so I am

yeah traitor I'm sure I'm
done you know it's bigger

it's bigger though it's
not many I want to call

it many and so it looks
like it's about the size

of the laptop have you
held one have you seen it

in person oh I have yeah
the I have held it in my

hand the question is are
the are the technical

specification right sure
sure yeah don't worry

okay yeah they well they
have it in photographs they

have it next to the laptop
and it looks roughly

the size of the laptop
it looks like a giant

pancake it's it's definitely
what I appreciate

is they're not ripping
off like the Mac studio

design or anything this
is a pretty unique design

it looks more like a game
console than anything

else your PlayStation 2
reference is actually

pretty pretty spot on
I would say my biggest

question is where are
the fans on this thing

because again I use my
GPU as a heater I need

fans on it to distribute
the heat throughout the

whole room so that while I'm
training neural networks

I can also be heated this
winter that's a very

important feature so I'm
curious how this thing uh

thermally conducts it's
thermal yeah I was just

because um it's pretty thin
you know it's it's kind

of like a two-story um
it's got a smaller bottom

than a in a bigger top
it's a little mushroom

shake and it was a little
bit confusing like where

they're putting all the
fans and everything on it

it's gorgeous I love it
um it reminds me of like

the old sun microsystems
boxes I don't know if

you remember any of those
from the early internet

days okay no never mind
um my biggest complaint

because I was telling you
before we started recording

is for roughly the same
price it's almost cost as

much as the same configuration
as the laptop but

you're not getting the
screen you're not getting

a keyboard you're not
getting a touchpad uh what

you are getting is about
25 more watts of power

I should say that the
laptop is 140 watt power

supply and this puppy is
165 watt power supply I

still think that that's
a little bit on the low

end from my personal taste
because my 3090 right

now is literally taking
up 350 watts of power so

it's already running
double the amount of power

um that this thing can
produce now obviously this

is a more modern chip it's
gonna do better it's a

it's more efficient chip
than my guy but you're

still stuck within that
power envelope so it's good

to know that um with the
laptop you're in the 140

watt power envelope with
this guy you're in the

165 watt power envelope
and these numbers matter

like physics matters you've
got to shove electricity

into these things in order
to make them go fast

um so it's it's pretty
cool to have this box out

I don't know who's gonna
buy one um I hope that

you're able to make a lot
of money on your Mac mini

I'm gonna be so sad that
you're not gonna be my

Mac mini brother anymore
I'm just gonna be out here

Mac miniing by myself but
I hope you're able to

get at least 10 grand
for your Mac mini so you

can buy two of these things
I did look I did look

and I did see that I
should be able to make a

few hundred dollars off
of you really yes okay I

don't know did you see
did Miguel say he's gonna

get the six pack of these
you know I don't know I

didn't I didn't see I
should a six pack of these

uh dev workstations that's
gonna be pretty awesome

yeah I'm I'm pretty jagged
I can't I can't argue

it's just a lot of money
it's a lot of money I'm uh

I think you know for me
obviously we'll keep my

Mac book air because I do
iOS development I need

a Mac around like you
cannot get around it and

the m1 still it's still
great device I don't know

I know it's still the
rumored like OLED um Apple

laptops coming out in a
couple weeks oh which by

the way a correction for
last week so the current

rumor is they are touchscreen
by the way so oh I

don't know did they say
whether the new ultra is

a touchscreen or the ultra
is a touchscreen yeah

yeah I figured as much
because Microsoft laptops

have they're all they're
all the touch sense yeah

ultra whatever it is so
um I think I said on the

podcast that it wasn't
going to be a touchscreen

Mac and it hasn't been
announced so this is not

official but the current
rumor mill is it is

actually a touchscreen Mac
so I was like maybe it

is time for me to upgrade
but then like look at

my cute little m1 I'm like
you're such a trooper

little m1 I don't think
I could ever sell you or

get rid of you like if
I ever drop you down

a six-story building
or something like that

accidentally then maybe
I will upgrade and fight

you're right yeah it's
hard to get rid of the

thing it's been so much
value for the buck yeah

I'm I'm super jazzed on
it this my current plan

and you know what I really
do like my my my Mac

mini and it's really great
I just actually don't

even know if I've taken
as much advantage of it

as I as I should then no
no I know if I will take

advantage of the the dev
box or not but we'll see

at least and I'm excited
to go through this journey

of where I'm at so the
the interesting part is

just like I went through
tons of transitions

with Thunderbolt 5 and this and that okay

well I go through all
this again with all my

crazy stuff I will say
this so I'm also reducing

my plan is to reduce my
ultra wide I'm getting

rid of my ultra wide okay
I'm convinced by John

Galloway that this $200
Dell 4k monitor that he

got on big day deals was
what I should get because

he said it was good and I
said I don't have Amazon

Prime so you're gonna need
to buy it for me wink

free free monitor um so he
said he said so so it's

it's come into the house
which is good which is

nice and getting rid of
that I want to like redo

my setup yet again obviously
but um it just it's

really tricky okay I
don't I have never really

felt like like I think
when I was building my

custom built tower
desktop PCs because I was

designing what it was I was
understanding the ports

I was preparing in it like
it was perfect I don't

know if I have rather
perig monitor setup but I

do feel when I had two
dual screens it like felt

correct like a dual screen
monitor setup it felt

correct for a while but
I do that yeah but I do

feel like I really want to
trim it down that being

said Frank Ruger I could
be convinced into an

ultra because it has the
other monitor right I

would have another monitor
that's there that I

could use and it coordinates
with this new 27 inch

4k monitor side by side
like I think about it you

know what I mean yeah I
don't have to get a stand

for it I gotta do something
I just think that that

that box will look so
sleek just sitting there

we'll see I don't know
who knows what I'll do

in general but that's right
at least where I'm at

right now and I'm real
excited I feel like you

know I've always been a
Windows person and when

the Mac mini came out the
size of it the power of

it spoke to me and so does
this that box it really

speaks to me as a as a
aesthetics developer mini

PC that I want and I
think I'm gonna get it so

we'll see oh boy well I
know I'm gonna send you a

bunch of models to try on
it because I'm gonna be

curious how fast it is so
I'm gonna make you test

the few things some remote
desktop access into

your own private account
that's something nice

make it into a proper
workstation server yeah

oh we didn't mention the
most important part of

the laptop before we go
I just have to say don't

call it make safe but they
added make safe to the

laptop it is a wild weirdest
little USB-C connector

you've ever seen in your
life and no one believes

it works but I've seen
video online of people

fake tripping over it
it seems to work fine

come on apple catch up because it's pretty

cool it's a pretty cool
design they did there

with the USB-C because
it's USB-C and it's make

safe but it's not safe copyright trademark

apple it's a it's the
world's first magnetic

USB-C I have one right
here yeah boom there it is

and it's over yet yeah
have you proven it I have

shipped over it many
times and it does work it

does work yes and it snaps
right in it's it's very

very nice so nice yeah
it's absolutely delightful

all right that's gonna
do a people for this

week's emerge conflict
go spend all your money

asking me service devices
I am super stoked

super good but that's
gonna do for this one so

until next time jay's on
to magnow and i'm frank

rueger thanks for watching
and listening peace