[{"data":1,"prerenderedAt":63},["ShallowReactive",2],{"leap-week-from-zero-to-production":3,"leap-week-from-zero-to-production-next":48},{"id":4,"slug":5,"vimeo_id":6,"description":7,"tile":8,"length":9,"resources":10,"people":10,"episode_number":11,"published":12,"title":13,"video_transcript_html":14,"video_transcript_text":15,"content":10,"status":16,"episode_people":17,"recommendations":32,"season":33,"seo":47},"0baede33-974c-4343-abad-3cea928c8112","from-zero-to-production","1176293308","Spin up a backend, connect a frontend, and deploy to production - all in one session. Featuring Railway.\n\n","14919273-9a84-4b2a-a651-20a11144d00d",38,null,4,"2026-03-27","From Zero to Production","\u003Cp>Speaker 0: Hi, everyone, and welcome to Leap Week. My name is Lindsay, and I'm an engineer on the direct us team where I mostly work on integrations and developer experience. For day two of Leap Week, the theme is build it, run it, sell it. The idea is to show how quickly you can go from an idea to a real production application using modern tools. We'll be focusing on the first part of that story, build it.\u003C\u002Fp>\u003Cp>For this session, we're going to build a fictional AI automation agency. The idea is that this agency helps companies implement AI workflows for things like marketing, automation, research, sales outreach, and internal knowledge assistance. So by the end of the session, we'll have a working back end for this agency, generate a front end website for it, and deploy the whole thing live. But before we can build anything, we need somewhere to run our back end and infrastructure. I'm joined today by someone from the Railway team.\u003C\u002Fp>\u003Cp>I'll let them introduce themselves and tell us a little bit about Railway before we start building.\u003C\u002Fp>\u003Cp>Speaker 1: Awesome. So, thank you so much for having me. My name is Mohammed. I'm a general engineer at Railway. And at Railway is I would say in a nutshell, it's a all in one intelligent cloud provider.\u003C\u002Fp>\u003Cp>You can deploy pretty much anything, databases, back ends, front ends, queues, whatever. Like, you can just deploy it. You can just run it. And, yeah, essentially, we allow you to deploy anything, and we really focus on making it easy for you to go from zero to deploy it as easy as possible. Yeah.\u003C\u002Fp>\u003Cp>So excited about what we'll build today today.\u003C\u002Fp>\u003Cp>Speaker 0: Awesome. So the first step in getting any project off the ground is gonna be your infrastructure. And instead of spending hours setting up servers and databases, Railways gonna let us start from a working template. For this demo, we have a template we've already created. It's a pretty simple one.\u003C\u002Fp>\u003Cp>It's gonna have everything you need to get started. We do actually have another version as well that launches with CMS collections already started if you want a starting point. But for this, we are going to use our blank one. So I'll go ahead and launch that template here.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. So a little bit of context. Templates essentially encapsulate a set of infrastructure components, all the configuration. And here, we essentially have, like, a database set up, post press red as a bucket for object storage, as well as directives, which runs our back end. So, like, directives here would be, a server that we actually use.\u003C\u002Fp>\u003Cp>And now, like, everything here is neatly on the railway canvas. You can see everything grouped under, like, the directors group. And, essentially, like, here, you have a project, and a project is where you can have essentially everything deployed. So, like, in this example here, you have practice running. Maybe you will have, like, the front end, so that will be a separate service.\u003C\u002Fp>\u003Cp>And that's pretty much all we need to get started. So, like, all you need to do is you can sign up for free, and then you can go to the link to deploy the template. You click deploy now. You can either deploy to a new project or an existing project. If you choose, like, the path we're going through right now is deploying to a new project, this is the experience you'll get.\u003C\u002Fp>\u003Cp>And now you can see, actually, it's like we have the difference for versus, they're being deployed. So, like, Post JS, that's the database. And then we have Redis and as well, we have, DirectThis, which is also just cute. So, like, one thing after another will be deployed. So, like, in a, hopefully, a minute or two, everything should be ready, configured.\u003C\u002Fp>\u003Cp>You don't even need to, like, touch an editor, and it just works. So that's pretty much yeah. That's pretty much the overview for templates.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. So it looks like\u003C\u002Fp>\u003Cp>Speaker 1: Like, this is also deploying I see it's so this one is actually deploying from a Docker image, and this is the official one, for directors. I assume this is the latest version. Right?\u003C\u002Fp>\u003Cp>Speaker 0: Yep. This is our latest version.\u003C\u002Fp>\u003Cp>Speaker 1: Yep. We make sure\u003C\u002Fp>\u003Cp>Speaker 0: to keep this updated.\u003C\u002Fp>\u003Cp>Speaker 1: That's awesome. But, yeah, if, let's say, for example, you deploy and you want to get the latest version, of, let's say, Directus. If you actually go to settings, you'll be able to see the source image. There's like automatic updates. You'll actually be able to see if there's an automatic update.\u003C\u002Fp>\u003Cp>You just click apply. You deploy it. It just works if you want like the latest version. So it should be very easy to set up, but so far we can now we see, we have the database ready. Red is ready.\u003C\u002Fp>\u003Cp>The bucket is created, and now the last component direct list is being deployed, which hopefully should be ready Yep. Very soon. And we're live.\u003C\u002Fp>\u003Cp>Speaker 0: We'll click that. So in just a couple minutes, we've gone from nothing to a running back end with a database, API, and admin interface. So as part of our setup process, we'll create our first admin user here. Give it a password.\u003C\u002Fp>\u003Cp>Speaker 1: So I assume this happens as the first initial step. Is this correct?\u003C\u002Fp>\u003Cp>Speaker 0: Yes. Yep. Okay. When you first set up, it'll ask you to create your admin user. And here we go.\u003C\u002Fp>\u003Cp>We're in our, UI here. And to get started, we are gonna do a little bit of, setup for our AI assistant because we are going to be using AI to create most of this. So Directus has a built in AI assistant that can help you. It can generate schema content, and it has knowledge of what's going on in your instance. So if you go into settings here, we'll give it a key.\u003C\u002Fp>\u003Cp>Oops. Let me grab my key.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. I think you just said\u003C\u002Fp>\u003Cp>Speaker 0: One password. Gotta have that security.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. And then we are also going to set up our MCP for later because we'll be using that when we do our front end.\u003C\u002Fp>\u003Cp>Speaker 1: So here, this is, like, the Directus MCP or, like, this is an MCP server for our deployed direct the Directus instance?\u003C\u002Fp>\u003Cp>Speaker 0: Yes. It is. Yep. So our cursor, which we'll use to do the front end, will be able to access all of our data here. And it'll depend on the permissions we give it, what it will be able to do.\u003C\u002Fp>\u003Cp>But we also have this in app AI assistant here. So we're gonna use that to generate our schema. So I already have some\u003C\u002Fp>\u003Cp>Speaker 1: So quick question. For someone who's not a 100 to 100% familiar, let's say, with all of the features, essentially, what we're doing is now we have a like, our complete back end deployed running. And part of this back end, you can actually expose an MTP server, and then I can connect to this MTP server via call code, code x cursor, whatever coding agent of my choice, and then just tell it to do stuff for me. That'll be reflected in my live deployed instance, and now it's just that's it.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Yeah. We'll see that a little bit later, but we're gonna do something similar here. We also have our AI assistant that's in the UI here, and we are gonna use that to build our back end, our collections, our schema. So right now, we have no collections created.\u003C\u002Fp>\u003Cp>We don't have any data tables, basically. So I'm gonna go ahead and prompt it with a prompt that I created earlier, and it's going to create the structure of our back end.\u003C\u002Fp>\u003Cp>Speaker 1: And the idea is, like, a collection would be, like, an object, right, for something.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. So it's gonna be, like, your data tables. So it'll have, like, an overlying day data table where we'll have one called site settings, and that's gonna be the settings for the website. We'll have one for case studies.\u003C\u002Fp>\u003Cp>Speaker 1: Mhmm.\u003C\u002Fp>\u003Cp>Speaker 0: That's gonna have case studies for our AI agency. Actually, I can show you the prompt here a little closer. So we've we're creating our collections, and we're giving it different fields. So we're giving it, like, a tagline, a headline. We're gonna give it different services that this AI agency does, give it some fields there, and we're gonna create playbooks and case studies out of those, so things you might need in an AI agency marketing website.\u003C\u002Fp>\u003Cp>And the AI on our end is going to do that work for us. So we've told it in our prompt what we want. We didn't really tell it what type of fields they need to be, if they're strings or numbs or any of that. It'll infer that if we want. And, actually, one nice thing is if you don't know what schema you want, you can talk it through with the AI assistant.\u003C\u002Fp>\u003Cp>It can give you suggestions of things that you might wanna add for a data table. So it's gonna ask questions too back to the user. So it's saying it's ready to create these, and it's saying, do we wanna proceed with its defaults? Do we wanna review each one and decide? Or do we wanna have, like, it be super minimal?\u003C\u002Fp>\u003Cp>I'm gonna say just create it. Let's see what it comes up with. It's asking how we should store the icon. So let's just say a single file upload. How do we wanna store the pricing start?\u003C\u002Fp>\u003Cp>We don't really care about that. We'll just pick something.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: There's a\u003C\u002Fp>\u003Cp>Speaker 1: there's a recommendation. So I think we can\u003C\u002Fp>\u003Cp>Speaker 0: go with it. Yeah. It's collaborative. It's gonna ask you how you wanna do things. It's not gonna just bowl over your user and, like, do everything for you.\u003C\u002Fp>\u003Cp>It's it's not just an AI tool that's gonna do it for you. It is a human in the loop type of experience where you can make sure it's doing something you actually want it to do. And it's gonna tell you everything it's doing while it's doing it. So you can see here. It's planning it out.\u003C\u002Fp>\u003Cp>You can see the thinking if you wanna know, like, what it's thinking while it does it. And you can do the different models if you're you have a different one. We're currently just doing, GPT five. In those settings we set up earlier, you can change that out. Like, if you want something different going on, you can say which ones are allowed, because this UI is actually not just for your admin.\u003C\u002Fp>\u003Cp>You can allow other people here. So you can have other people with different access levels able to come in and modify your content once it gets created or your data tables. So you can let them use the AI assistant, and you can set what permission levels they have. So it's gonna ask for different tools like creating the collections if it can do it before it does it. You'll see here's our data models now.\u003C\u002Fp>\u003Cp>It's created a couple of them. We've got our case studies one. It's starting to add the fields into these. In the past, you would have to do this all manually and create collections. Now you can just use the AI to do it without having to worry too much.\u003C\u002Fp>\u003Cp>So it's a lot less manual work.\u003C\u002Fp>\u003Cp>Speaker 1: But we could have done the same thing by using the MCP thing. Right?\u003C\u002Fp>\u003Cp>Speaker 0: Yep. You can. Yeah. We just wanted to give people an in app way to do it a little faster and simpler for people who maybe aren't don't have a data model or AI model they like to use outside of here.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Definitely makes sense. It's just like in my mind, I really like the flow of using some like, I use Claude a lot. So when I use Claude, I'm like, okay. I want a complete end to end flow.\u003C\u002Fp>\u003Cp>Like, what what would it look like if I don't leave cloth? Can I actually do the stuff I wanna do? And it seems like the answer is yeah. But there's also And\u003C\u002Fp>\u003Cp>Speaker 0: that's actually that's what I like to do as well as I'll go in cursor, and you could start the whole front and back end directly in cursor using the direct SMCP, and it can create it in here for you. We're gonna do the front end in Cursor just so we're seeing that too a little bit, but I wanted to use a little of the AI assistant as well.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah. I'm actually in the process of making sure that our experience of getting started to be super seamless. So we already have, like, a regular CLI. We have, like, Asian skills, and, we're going to essentially keep shipping more, features through the CLI.\u003C\u002Fp>\u003Cp>So, like, it should be able to you say, like, oh, I wanna deploy direct us. It should be able to go through the templates marketplace, find the most up to date template deployed for you, and then it should be able to also figure out, like, the deployment is finished. Let me continue. And then you just have, like, a complete workflow where you don't even have to go to a templates page. So that's kind of, like, what's\u003C\u002Fp>\u003Cp>Speaker 0: That's awesome.\u003C\u002Fp>\u003Cp>Speaker 1: In the works. But yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. I'm excited for that. That'll be good.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. That's I think it asked a question at the end.\u003C\u002Fp>\u003Cp>Speaker 0: Yep. So now it is done. It's gonna, like if you've used this model, it constantly asks you other things it wants to do, but it is done. It's created the fields. So we've got that part done.\u003C\u002Fp>\u003Cp>Now we're gonna give it some fake content in these fields. So that's the database table. And now we're gonna fill these tables with actual, like, fake data. So I'm gonna paste that in. So when we go into here, you'll see no content yet, and it's gonna start filling those in.\u003C\u002Fp>\u003Cp>But you can see, like, these are the fields. So if a nontechnical user came in and wanted to give it a new name or a new tagline for their website, they can do it in here without having to touch the code at all. So it's a nice way to work with both your front end team and your back end team and your content team and not have, people who are super technical not able to get things done. While this is doing it, we are actually gonna get finished setting up for our MCP, actually. So we're gonna go here in our docs.\u003C\u002Fp>\u003Cp>We have a little section for setting up the MCP in different tools. We're gonna use cursor. So I'm gonna click add to cursor here. Nice easy button, and it will set us up into our settings. And we are going to go grab our URL from here.\u003C\u002Fp>\u003Cp>So this is our directest instance URL, and we'll pop that in here for our MCP setup. So it's just your instance URL slash MCP, and then you're gonna wanna set up a token. So let's make sure it has the permissions it needs. When you're connecting to external tools like cursor, it's always a good idea to scope your permissions so that it's not able to completely nuke your whole setup. You know, AI likes to be overeager.\u003C\u002Fp>\u003Cp>Speaker 1: So Yeah. Yeah. It's like, let's start all over.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. We need a fresh start. We we recommend setting up a token for it specifically and not using your admin token. So you wanna go to your user roles here. You'll create a new role, name it whatever you want, and you'll add a policy.\u003C\u002Fp>\u003Cp>We're gonna give it view only policy. So you'll add your collections, give it read only access. That way it can't, like, update or delete or anything when it's in the MCP. If you want the MCP to do that, you can totally give it permissions to create, read, and update, and do whatever you want with it. But I recommend giving it, like, the lowest level of access at the start so that, you know, it's not gonna go in and break your whole setup here.\u003C\u002Fp>\u003Cp>Mhmm. So we've got our policy. I'm gonna create a new user too to grab a token from. So if we scroll down to our tokens, we're gonna generate it and save. And then we'll go back in here, and that's where you put in this bear here, authorization code.\u003C\u002Fp>\u003Cp>You put that token, click install, and that will install our direct as MCP encursors. So now we're ready to go for that when this is finished. So let's go check on it.\u003C\u002Fp>\u003Cp>Speaker 1: So what we've done is create a role, and then the role has the token?\u003C\u002Fp>\u003Cp>Speaker 0: Yes. So the the user is what who has the token. So this MCP user has the role that we created. I created it all within the same page, but this person has the role, and then they have the token. Mhmm.\u003C\u002Fp>\u003Cp>So if you go into our settings here and look at the MCP role, we've got our view only policy that we set up Mhmm. For our different collections. And the users in the role is that MCP guy. So now it looks like it did finish the content. If we go back into our content page, now you can see it's filled it in with our sample data.\u003C\u002Fp>\u003Cp>I gave it a few guidelines of the type of things I wanted it to have, but, otherwise, it just kinda came up with this on its own, sample content. So it's given us a starting point. So now that we've got some sample content, we are going to go back in to cursor. I'm gonna grab this really long prompt I've created telling it basically we are an AI agency. We're creating our website.\u003C\u002Fp>\u003Cp>I gave it I told it to use next. I also made sure to tell it to use the Directus MCP server. This might take a minute, so I'm gonna go ahead and get it started. We are in a completely empty folder here, so I'm gonna have it do all of the coding for me. Let it create what we need here.\u003C\u002Fp>\u003Cp>It's gonna make sure to look at our schema via the MCP. So we want it to not guess on what structure we have of our data. We want it to actually look at it. Like we said, we could create the data directly from Cursor via the MCP. You would have to make sure you give it, the right permissions.\u003C\u002Fp>\u003Cp>So that that might have to be an admin token in order to generate new schema with your MCP. So if you're comfortable doing that, you totally can.\u003C\u002Fp>\u003Cp>Speaker 1: Interesting. So once that's done, we'll have something that's working like a front end running locally that's fully functional, pulling data from our deployed direct us instance. Correct?\u003C\u002Fp>\u003Cp>Speaker 0: Yep.\u003C\u002Fp>\u003Cp>Speaker 1: Okay.\u003C\u002Fp>\u003Cp>Speaker 0: Yep. And any changes you make in direct us would be reflected in your local front end that you've got here. And then as the final step, we are going to do to deploy this front end to railway so that you have a full production application front end and back end.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. I have cursor set up, so I could just automatically runs everything. I don't, like, manually approve stuff. I just\u003C\u002Fp>\u003Cp>Speaker 0: I don't trust it. Yeah. I gotta make sure I know what it's doing. I've been burned before.\u003C\u002Fp>\u003Cp>Speaker 1: I don't know. The the models are they keep getting better and it's like, I'm like, I would've done the same thing. So I don't know. I feel like it's just faster and you can just like, you know, sit back, relax, grab your favorite drink while it gets to work. You come back to it and you're like, woah, this thing actually works.\u003C\u002Fp>\u003Cp>So Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: I think I'm a bit of a control freak. I can't let it just just\u003C\u002Fp>\u003Cp>Speaker 1: go. I'm telling you, like, once you let go and you see, like, it actually works and it's like, I think, I think you'll be plenty surprised. Then you're like, wow, that's actually a really fun. Cause like now, you know, if let's say for whatever reason, someone brings you on Slack, you can play for guys. Like, you come back to us, like, oh, I forgot to approve.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. That it's done.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah. I I I think, it's a much more fun experience when you just let it kinda do its own thing without you steering it. But I do understand the, appeal of, you know, wanting to see what it does. But also, like, we're starting from scratch, so there's little, like, room for error, I would say.\u003C\u002Fp>\u003Cp>But yeah.\u003C\u002Fp>\u003Cp>Speaker 0: There's nothing for it to break. It's creating it from scratch. So\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: We're pretty comfortable letting it just do what it wants to do here. So it looks like it actually has gotten the types if you look. So it pulled this from our MCP. It checked in, Directus, and it got our back end and what types we have there. So that's good.\u003C\u002Fp>\u003Cp>Looks like it's I did tell it also to create a little connection a net little connector to direct us here, a fetcher.\u003C\u002Fp>\u003Cp>Speaker 1: So it\u003C\u002Fp>\u003Cp>Speaker 0: looks like it's done that, getting different gets to get the data.\u003C\u002Fp>\u003Cp>Speaker 1: And it will be able to pull in our deployed, like, like, URL and actually set it as an environment variable?\u003C\u002Fp>\u003Cp>Speaker 0: Yes. It will. So I it might make me make the e n v, but I've got it. It's supposed to use the direct URL here, and it's supposed to use the token that we created. So, we'll see.\u003C\u002Fp>\u003Cp>I've done this a couple times just to see how it would do, and it it doesn't always know to grab the URL. If it's really smart, it can grab it from the cursor settings, and it can get the token and the URL, but it doesn't always manage that. You'll notice I'm in auto mode, so it's a different model each time, so it might not be a very smart model. But we at the end, we'll give it the proper e n v variables, and it'll be able to connect to our deployed instance.\u003C\u002Fp>\u003Cp>Speaker 1: Excellent. Yeah. Honestly, again, I I like my to choose my own models, You should use, like, a, you know, g p three five point four or, like, an Opus 4.6 and because I don't know. I don't like that, I would say I wouldn't trust the auto. Like, which one would affect?\u003C\u002Fp>\u003Cp>Like, I wanna be I want a consistent experience. So then if, like, anything starts feeling weird, then I can blame the model. I was like, oh, you know, like, Opus is acting really dumb lately, that sort of thing. So\u003C\u002Fp>\u003Cp>Speaker 0: It it does just feel like from one day to the next, though, even if you have a model that you know is good, it it doesn't always behave exactly the same.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Sometimes it feels like, you know, you're you're just rolling the dice. Sometimes it's like, woah. This is it. Like, this is the next big thing.\u003C\u002Fp>\u003Cp>I was like cause like, sometimes I, I, I asked it a question and I wanted it to check against like, you know, our internal, GitHub repo. And it's like, Hey, do we support this thing? It was like, yes, you do. And I told her like, oh, show me the code. It's like, after checking, apparently, this is not supported.\u003C\u002Fp>\u003Cp>I was like, come. Yeah. And this is like the smart and I told it, like, oh, you know, UltraThink, you know, work really hard, but it's like, I guess sometimes it's just, you know, not that type of day. But I think here it's like it's it's it's working well, based on what I'm saying.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. It looks like it's getting something.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: It's messing around with the configs now.\u003C\u002Fp>\u003Cp>Speaker 1: Interesting. What is what's it doing to the Next. Js config if we open it up? Just out of curiosity. Technically, you know, we're we're not supposed to look at the code, you know, and, because, you know, I was just shipping it, but I'm also curious.\u003C\u002Fp>\u003Cp>Okay. It's setting, like, the assets URL, I assume, for, like, files and stuff. That's pretty awesome.\u003C\u002Fp>\u003Cp>Speaker 0: Yep. And while it does that, I'm actually gonna grab my last prompt, which is a prompt to have it, push to GitHub so that we can get it ready for deploying the front end. I'm gonna grab that. Go back in here. I think it's almost done.\u003C\u002Fp>\u003Cp>It feels like it's it's doing the read me. So\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Probably at the end. But yeah. I would say, there is actually a like, you you can definitely, you know, start with this workflow. It's totally fine.\u003C\u002Fp>\u003Cp>But if you have, like, the AOS CLI setup, it could just deploy it to your existing project. And this way you don't even have to worry about git being installed, get, having a GitHub repo or anything. But when you have it with git, well, the benefit is, anytime you make, changes through the front end, it will just automatically deploy. So it's like it's it's all trade offs based on, you know, how, structured and organized you wanna be. But yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. That makes sense. Alright. We did not create our E and V, so we're gonna create that real quick. And\u003C\u002Fp>\u003Cp>Speaker 1: I would say that's the responsible thing for it to do.\u003C\u002Fp>\u003Cp>Speaker 0: Yes. I don't didn't really like that it created its own ENV and, like, grabbed my things from the MCP. That was a little, uncomfortable. Yeah.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. It then gets better this way.\u003C\u002Fp>\u003Cp>Speaker 0: Yes. Look at our URL. Alright. And then we're gonna run it locally just to see it working.\u003C\u002Fp>\u003Cp>Speaker 1: What does the static token do?\u003C\u002Fp>\u003Cp>Speaker 0: So that's the one we created with for the MCP. We're using it here too. Okay. So that is the permissions that we allow in Directus. So if you give it, like, no read permission, it won't be able to see the content.\u003C\u002Fp>\u003Cp>If you give it access to read, like what we did, we gave it access to read all of these collections, then it can read the data. So that's where, like, you probably want a separate one from your MCP. We're just in the interest of time having it Yeah. One that is view all for all of our collections because we don't need to make changes with the MCP. But you probably would have permission to update and delete with your MCP as well.\u003C\u002Fp>\u003Cp>So you probably want wanted a separate one for your front end. So you'd probably wanna create a user role for your front end that gives it different permissions. Otherwise, you can have it set up to use your public permissions, which controls what the API data is, available without authenticating. But it's safer if you do give it a token so that your front end is only able to grab what you want, not, like, everything. So let's see if it worked.\u003C\u002Fp>\u003Cp>Speaker 1: Moment of truth.\u003C\u002Fp>\u003Cp>Speaker 0: Okay. There it is. Looks like some images are broken, but we've got a full website here. You can navigate around. It's got some prompts.\u003C\u002Fp>\u003Cp>It's got a button to get in touch.\u003C\u002Fp>\u003Cp>Speaker 1: We've got a\u003C\u002Fp>\u003Cp>Speaker 0: full website here. Obviously, you'd wanna iterate on it, fix the images, and, do a little honestly, it looks great. It doesn't need too much design or anything even. So that's a good starting point. We are going to go ahead and say good enough.\u003C\u002Fp>\u003Cp>The images can get fixed later. We are going to deploy this. And, of course, I copied other things, so I need to go back and get my prompt. So like you said, there's other ways to do this, but we are gonna have it pushed to a GitHub repo. We're pretending we have other people who wanna manage this code, and we want it in some place that we would have access to it as a team.\u003C\u002Fp>\u003Cp>So we've have a GitHub repo we've already set up. It's empty right now, and we're gonna push it into here. So I'm having it create a readme that'll tell users, our other team members, basically, is in our example here. We're deploying to Railway. Here's what you'll need.\u003C\u002Fp>\u003Cp>It'll mention the EMV variables we created here, because we're gonna need those. I'm actually gonna copy them now, so I have them ready. And then it it's gonna make sure everything's building properly with the prompt I gave it, and then it will push to GitHub for us.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. And I noticed it said, like, even if you don't have, like, the environment variable set up initially, like, it should still deploy. It should still work. But once we actually set the these values, we'll be able to actually fetch the thing and have the same result we have locally, but actually have it live and deployed on a URL.\u003C\u002Fp>\u003Cp>Speaker 0: Yep.\u003C\u002Fp>\u003Cp>Speaker 1: It's updating the ReadMe.\u003C\u002Fp>\u003Cp>Speaker 0: Yep. There's our ReadMe.\u003C\u002Fp>\u003Cp>Speaker 1: I remember the times where I had you know, I used to write stuff by hand.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah.\u003C\u002Fp>\u003Cp>Speaker 1: Now I just be like, you know, just instructions, just lines, like update, read me, and that's it. You know? Just just figure it out the rest.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. I think it's gonna commit.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. I think it's already I believe I ran already a built. So, yeah, we see, like, the dot next directory. Maybe the build was under there. Yeah.\u003C\u002Fp>\u003Cp>It's running get the commands. Okay. I think it has committed everything here. So now\u003C\u002Fp>\u003Cp>Speaker 0: What did I do?\u003C\u002Fp>\u003Cp>Speaker 1: I think it's just the wrong link.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. It did. Okay. So we are gonna now go in here. We're gonna add a service.\u003C\u002Fp>\u003Cp>Speaker 1: Yep.\u003C\u002Fp>\u003Cp>Speaker 0: GitHub repository. And what did I call it?\u003C\u002Fp>\u003Cp>Speaker 1: Was it like AI agents or something like that?\u003C\u002Fp>\u003Cp>Speaker 0: AI agents. Not sure. Oh, boy. Hold on. I pasted it down here.\u003C\u002Fp>\u003Cp>Speaker 1: You should be able to see, like, all repos, but maybe you have too many. So, like\u003C\u002Fp>\u003Cp>Speaker 0: I have so many. I really need to clean it up.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah. That's me.\u003C\u002Fp>\u003Cp>Speaker 0: Okay. Hold on. I'm gonna\u003C\u002Fp>\u003Cp>Speaker 1: Give me to refresh. Yeah. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Just don't wanna have my mega list of repos up on the screen so one's.\u003C\u002Fp>\u003Cp>Speaker 1: But I think you might need to refresh the, so I tried to push the code. Right? Oh, it didn't.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. I don't know if it\u003C\u002Fp>\u003Cp>Speaker 1: pushed. I mean, I guess you need to stop. Yeah. See\u003C\u002Fp>\u003Cp>Speaker 0: It's getting me with the permission things. You're right.\u003C\u002Fp>\u003Cp>Speaker 1: I should\u003C\u002Fp>\u003Cp>Speaker 0: just let it do whatever it wants.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. But now it's I don't know if it's pushing it, and then it should be good. So now if you go to right way, I would say if, if you refresh, you should be able to see it. Yeah. We have code.\u003C\u002Fp>\u003Cp>Speaker 0: The code is now here.\u003C\u002Fp>\u003Cp>Speaker 1: Yep. Now when you add, we might need to like refer, I think it should just fetch, but you can try.\u003C\u002Fp>\u003Cp>Speaker 0: Does it not like my\u003C\u002Fp>\u003Cp>Speaker 1: Maybe just try refreshing, like, the, entire Page. Dashboard. Yeah. Yeah. Just in case.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Okay. And Okay. Let's try again.\u003C\u002Fp>\u003Cp>Speaker 1: Interesting. I mean, if you want, what you could do is maybe it's just an issue with, like, get up not pulling. So, like, you can copy the full URL. And when you add, you can just paste that URL.\u003C\u002Fp>\u003Cp>Speaker 0: Okay. The it looks like it found it like that. So let's\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. So that is how you connect it, and then\u003C\u002Fp>\u003Cp>Speaker 1: we're gonna just need to hit deploy. Yep.\u003C\u002Fp>\u003Cp>Speaker 0: Yes. But I wanna put it up by the other ones first. Yeah. And that was\u003C\u002Fp>\u003Cp>Speaker 1: in there.\u003C\u002Fp>\u003Cp>Speaker 0: Yes. So I wanna also add our e and v variables before we get started.\u003C\u002Fp>\u003Cp>Speaker 1: So they're actually automatically inferred, like, at the bottom. Oh. You can see them. You were right. Suggest variables, and then you can update these values and click add, then you're good to go.\u003C\u002Fp>\u003Cp>Speaker 0: There you go. So let me grab where I pasted them over here.\u003C\u002Fp>\u003Cp>Speaker 1: You can also have, like, go the route you were going through, like the raw editor, copy and paste that other thing, but just just wanted to point it out. Because, like, to me, it's like, it's one of those things that when you don't know it, you're like, oh, that's neat. That's kinda how I always felt about it the first time I tried it.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. You're right. That is cool that it already kind of pulled up what we needed.\u003C\u002Fp>\u003Cp>Speaker 1: But,\u003C\u002Fp>\u003Cp>Speaker 0: yeah, you could also open raw editor and just paste in the whole EMV file that we had ready.\u003C\u002Fp>\u003Cp>Speaker 1: Yep. And then you can click add to add everything.\u003C\u002Fp>\u003Cp>Speaker 0: Yep. So these this is the token we have in our local one, and then this is the URL of the same thing in Railway. You guys can also do internal URLs. Right? Is that something we could use here if we wanted to use the local URL\u003C\u002Fp>\u003Cp>Speaker 1: of this? So you can do, like, a dollar sign, two curly braces, and then you should get auto complete for, like, the URL. So, actually, you will be able to see a line where they're both connected if you wanna try it out. And then what will happen is, now whenever if the URL changes, it will just automatically update the other one as well and redeploy it.\u003C\u002Fp>\u003Cp>Speaker 0: Okay. So that's like using the internal\u003C\u002Fp>\u003Cp>Speaker 1: URL reference variable. So I I believe you the way it would work is you just do, so the on directives and the variables themselves, like, if you go to variables. I believe what's the yeah. I guess it's public URL is the value that we will want. So wait.\u003C\u002Fp>\u003Cp>If you go now to the AI agency service and then go to the next public directors URL, because that's the URL value you wanna set. Right? So that more menu and then edit. And then if you do, dollar sign and then double curly braces, you'll be able to actually pull in so I believe the name of the service is directed. So you just do direct us, and then it's gonna be dot yep.\u003C\u002Fp>\u003Cp>Do you see? And then you just choose the public URL Yep. From this list. Got it. So now when you choose it, it's just gonna automatically give you the right URL and it will just be able to read it.\u003C\u002Fp>\u003Cp>So if the other one updates, you just do it. But for this, you're gonna need to hit redeploy again. So it will take some time. So I'd say we can do this after you can cancel. But yeah.\u003C\u002Fp>\u003Cp>And I would say there's actually another cool feature because might as well, if you go to settings, like, okay. It's done. So we can actually test the thing. But we can update, like, the domain to, like, a actual domain because you can now buy domains on railway.\u003C\u002Fp>\u003Cp>Speaker 0: Nice. Okay.\u003C\u002Fp>\u003Cp>Speaker 1: Yep. So this will give to you just a domain for, the front end itself, and it will have, like, a .up.railway.app at the end. You'll need to specify a port, which I'm not sure what it is on. I I think it should be 3,000. Like, 3,000 should just work.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Yeah.\u003C\u002Fp>\u003Cp>Speaker 1: And if we visit it, might be the incorrect port. You could try eighty eighty as well because that's what I was suggesting in the beginning. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: There we go.\u003C\u002Fp>\u003Cp>Speaker 1: There we go.\u003C\u002Fp>\u003Cp>Speaker 0: So there is our front end deployed.\u003C\u002Fp>\u003Cp>Speaker 1: Yep. And we have a live app.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Awesome. So that was pretty fast. We went from nothing to a fully deployed front and back end here. Let's see.\u003C\u002Fp>\u003Cp>Was there anything else we should know about Railway and getting this set up here?\u003C\u002Fp>\u003Cp>Speaker 1: I mean, if you hit deploy now, it will just redeploy, and you'll actually be able to see on the canvas. There's, like, an arrow, and this, service, like our front end is, referencing the direct us. And that's how you have, like, a connection between the two. And, yeah, other than that, it's like, you can just have everything in one place. Again, like, if you go now to the AI agency service and you click on settings and you also go to networking, from the right.\u003C\u002Fp>\u003Cp>Yeah. And then you can actually choose a custom domain. So when you click on it, you should actually be able to buy a domain straight from here. So like, if you come up with that domain, you can just do it from here as well. Or if you go to like ferry.com\u002Fdomains, you can do it.\u003C\u002Fp>\u003Cp>But yeah, I would say that's kinda like you now have fully like everything in one place, but you know, took us not a lot of time, and we have, like, a fully, fully deployed working back end, front end. Just, yeah, everything is in one place.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. So why don't I recap what we did? We started with a simple idea launching a fictional AI automation agency. We launched infrastructure on Railway. We generated a structured back end with Directus.\u003C\u002Fp>\u003Cp>We populated it with fake content. We generated the front end using Cursor. We deployed the whole thing back to Railway. That's the build it part of today's theme. The key takeaway is that when interfaces can be generated quickly, the back end data model becomes the strategic layer.\u003C\u002Fp>\u003Cp>If someone watching wants to try this themselves, what's just the easiest way to get started on the railway side of things?\u003C\u002Fp>\u003Cp>Speaker 1: The easiest way, I would say going to the template you shared would be the easiest place. And if they, you know, wanna use direct this. And also, by the way, I forgot to mention. So if you actually, like, close the, service here, there's like an agent in the top right corner as well, where you can actually also ask questions about your stuff, like actually ask it to make changes and it should be able to make changes, directly. So I believe if you just say like, oh, update, make this update, it should figure things out on its own.\u003C\u002Fp>\u003Cp>Speaker 0: Awesome. So you guys got some AI in here too?\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Yeah. In case someone wants everything in the dashboard, they can, but also if they want, the, to use whatever their coding agent of choice, they should be able to do it. But yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Cool. Well, thank you for joining us. I hope everybody has a good rest of your Leap Week.\u003C\u002Fp>\u003Cp>Speaker 1: Sounds great. Thank you so much for having me.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Thank you for joining us.\u003C\u002Fp>","Hi, everyone, and welcome to Leap Week. My name is Lindsay, and I'm an engineer on the direct us team where I mostly work on integrations and developer experience. For day two of Leap Week, the theme is build it, run it, sell it. The idea is to show how quickly you can go from an idea to a real production application using modern tools. We'll be focusing on the first part of that story, build it. For this session, we're going to build a fictional AI automation agency. The idea is that this agency helps companies implement AI workflows for things like marketing, automation, research, sales outreach, and internal knowledge assistance. So by the end of the session, we'll have a working back end for this agency, generate a front end website for it, and deploy the whole thing live. But before we can build anything, we need somewhere to run our back end and infrastructure. I'm joined today by someone from the Railway team. I'll let them introduce themselves and tell us a little bit about Railway before we start building. Awesome. So, thank you so much for having me. My name is Mohammed. I'm a general engineer at Railway. And at Railway is I would say in a nutshell, it's a all in one intelligent cloud provider. You can deploy pretty much anything, databases, back ends, front ends, queues, whatever. Like, you can just deploy it. You can just run it. And, yeah, essentially, we allow you to deploy anything, and we really focus on making it easy for you to go from zero to deploy it as easy as possible. Yeah. So excited about what we'll build today today. Awesome. So the first step in getting any project off the ground is gonna be your infrastructure. And instead of spending hours setting up servers and databases, Railways gonna let us start from a working template. For this demo, we have a template we've already created. It's a pretty simple one. It's gonna have everything you need to get started. We do actually have another version as well that launches with CMS collections already started if you want a starting point. But for this, we are going to use our blank one. So I'll go ahead and launch that template here. Yeah. So a little bit of context. Templates essentially encapsulate a set of infrastructure components, all the configuration. And here, we essentially have, like, a database set up, post press red as a bucket for object storage, as well as directives, which runs our back end. So, like, directives here would be, a server that we actually use. And now, like, everything here is neatly on the railway canvas. You can see everything grouped under, like, the directors group. And, essentially, like, here, you have a project, and a project is where you can have essentially everything deployed. So, like, in this example here, you have practice running. Maybe you will have, like, the front end, so that will be a separate service. And that's pretty much all we need to get started. So, like, all you need to do is you can sign up for free, and then you can go to the link to deploy the template. You click deploy now. You can either deploy to a new project or an existing project. If you choose, like, the path we're going through right now is deploying to a new project, this is the experience you'll get. And now you can see, actually, it's like we have the difference for versus, they're being deployed. So, like, Post JS, that's the database. And then we have Redis and as well, we have, DirectThis, which is also just cute. So, like, one thing after another will be deployed. So, like, in a, hopefully, a minute or two, everything should be ready, configured. You don't even need to, like, touch an editor, and it just works. So that's pretty much yeah. That's pretty much the overview for templates. Yeah. So it looks like Like, this is also deploying I see it's so this one is actually deploying from a Docker image, and this is the official one, for directors. I assume this is the latest version. Right? Yep. This is our latest version. Yep. We make sure to keep this updated. That's awesome. But, yeah, if, let's say, for example, you deploy and you want to get the latest version, of, let's say, Directus. If you actually go to settings, you'll be able to see the source image. There's like automatic updates. You'll actually be able to see if there's an automatic update. You just click apply. You deploy it. It just works if you want like the latest version. So it should be very easy to set up, but so far we can now we see, we have the database ready. Red is ready. The bucket is created, and now the last component direct list is being deployed, which hopefully should be ready Yep. Very soon. And we're live. We'll click that. So in just a couple minutes, we've gone from nothing to a running back end with a database, API, and admin interface. So as part of our setup process, we'll create our first admin user here. Give it a password. So I assume this happens as the first initial step. Is this correct? Yes. Yep. Okay. When you first set up, it'll ask you to create your admin user. And here we go. We're in our, UI here. And to get started, we are gonna do a little bit of, setup for our AI assistant because we are going to be using AI to create most of this. So Directus has a built in AI assistant that can help you. It can generate schema content, and it has knowledge of what's going on in your instance. So if you go into settings here, we'll give it a key. Oops. Let me grab my key. Yeah. I think you just said One password. Gotta have that security. Yeah. Alright. And then we are also going to set up our MCP for later because we'll be using that when we do our front end. So here, this is, like, the Directus MCP or, like, this is an MCP server for our deployed direct the Directus instance? Yes. It is. Yep. So our cursor, which we'll use to do the front end, will be able to access all of our data here. And it'll depend on the permissions we give it, what it will be able to do. But we also have this in app AI assistant here. So we're gonna use that to generate our schema. So I already have some So quick question. For someone who's not a 100 to 100% familiar, let's say, with all of the features, essentially, what we're doing is now we have a like, our complete back end deployed running. And part of this back end, you can actually expose an MTP server, and then I can connect to this MTP server via call code, code x cursor, whatever coding agent of my choice, and then just tell it to do stuff for me. That'll be reflected in my live deployed instance, and now it's just that's it. Yeah. Yeah. We'll see that a little bit later, but we're gonna do something similar here. We also have our AI assistant that's in the UI here, and we are gonna use that to build our back end, our collections, our schema. So right now, we have no collections created. We don't have any data tables, basically. So I'm gonna go ahead and prompt it with a prompt that I created earlier, and it's going to create the structure of our back end. And the idea is, like, a collection would be, like, an object, right, for something. Yeah. So it's gonna be, like, your data tables. So it'll have, like, an overlying day data table where we'll have one called site settings, and that's gonna be the settings for the website. We'll have one for case studies. Mhmm. That's gonna have case studies for our AI agency. Actually, I can show you the prompt here a little closer. So we've we're creating our collections, and we're giving it different fields. So we're giving it, like, a tagline, a headline. We're gonna give it different services that this AI agency does, give it some fields there, and we're gonna create playbooks and case studies out of those, so things you might need in an AI agency marketing website. And the AI on our end is going to do that work for us. So we've told it in our prompt what we want. We didn't really tell it what type of fields they need to be, if they're strings or numbs or any of that. It'll infer that if we want. And, actually, one nice thing is if you don't know what schema you want, you can talk it through with the AI assistant. It can give you suggestions of things that you might wanna add for a data table. So it's gonna ask questions too back to the user. So it's saying it's ready to create these, and it's saying, do we wanna proceed with its defaults? Do we wanna review each one and decide? Or do we wanna have, like, it be super minimal? I'm gonna say just create it. Let's see what it comes up with. It's asking how we should store the icon. So let's just say a single file upload. How do we wanna store the pricing start? We don't really care about that. We'll just pick something. Yeah. Yeah. There's a there's a recommendation. So I think we can go with it. Yeah. It's collaborative. It's gonna ask you how you wanna do things. It's not gonna just bowl over your user and, like, do everything for you. It's it's not just an AI tool that's gonna do it for you. It is a human in the loop type of experience where you can make sure it's doing something you actually want it to do. And it's gonna tell you everything it's doing while it's doing it. So you can see here. It's planning it out. You can see the thinking if you wanna know, like, what it's thinking while it does it. And you can do the different models if you're you have a different one. We're currently just doing, GPT five. In those settings we set up earlier, you can change that out. Like, if you want something different going on, you can say which ones are allowed, because this UI is actually not just for your admin. You can allow other people here. So you can have other people with different access levels able to come in and modify your content once it gets created or your data tables. So you can let them use the AI assistant, and you can set what permission levels they have. So it's gonna ask for different tools like creating the collections if it can do it before it does it. You'll see here's our data models now. It's created a couple of them. We've got our case studies one. It's starting to add the fields into these. In the past, you would have to do this all manually and create collections. Now you can just use the AI to do it without having to worry too much. So it's a lot less manual work. But we could have done the same thing by using the MCP thing. Right? Yep. You can. Yeah. We just wanted to give people an in app way to do it a little faster and simpler for people who maybe aren't don't have a data model or AI model they like to use outside of here. Yeah. Definitely makes sense. It's just like in my mind, I really like the flow of using some like, I use Claude a lot. So when I use Claude, I'm like, okay. I want a complete end to end flow. Like, what what would it look like if I don't leave cloth? Can I actually do the stuff I wanna do? And it seems like the answer is yeah. But there's also And that's actually that's what I like to do as well as I'll go in cursor, and you could start the whole front and back end directly in cursor using the direct SMCP, and it can create it in here for you. We're gonna do the front end in Cursor just so we're seeing that too a little bit, but I wanted to use a little of the AI assistant as well. Yeah. Yeah. I'm actually in the process of making sure that our experience of getting started to be super seamless. So we already have, like, a regular CLI. We have, like, Asian skills, and, we're going to essentially keep shipping more, features through the CLI. So, like, it should be able to you say, like, oh, I wanna deploy direct us. It should be able to go through the templates marketplace, find the most up to date template deployed for you, and then it should be able to also figure out, like, the deployment is finished. Let me continue. And then you just have, like, a complete workflow where you don't even have to go to a templates page. So that's kind of, like, what's That's awesome. In the works. But yeah. Yeah. I'm excited for that. That'll be good. Yeah. That's I think it asked a question at the end. Yep. So now it is done. It's gonna, like if you've used this model, it constantly asks you other things it wants to do, but it is done. It's created the fields. So we've got that part done. Now we're gonna give it some fake content in these fields. So that's the database table. And now we're gonna fill these tables with actual, like, fake data. So I'm gonna paste that in. So when we go into here, you'll see no content yet, and it's gonna start filling those in. But you can see, like, these are the fields. So if a nontechnical user came in and wanted to give it a new name or a new tagline for their website, they can do it in here without having to touch the code at all. So it's a nice way to work with both your front end team and your back end team and your content team and not have, people who are super technical not able to get things done. While this is doing it, we are actually gonna get finished setting up for our MCP, actually. So we're gonna go here in our docs. We have a little section for setting up the MCP in different tools. We're gonna use cursor. So I'm gonna click add to cursor here. Nice easy button, and it will set us up into our settings. And we are going to go grab our URL from here. So this is our directest instance URL, and we'll pop that in here for our MCP setup. So it's just your instance URL slash MCP, and then you're gonna wanna set up a token. So let's make sure it has the permissions it needs. When you're connecting to external tools like cursor, it's always a good idea to scope your permissions so that it's not able to completely nuke your whole setup. You know, AI likes to be overeager. So Yeah. Yeah. It's like, let's start all over. Yeah. We need a fresh start. We we recommend setting up a token for it specifically and not using your admin token. So you wanna go to your user roles here. You'll create a new role, name it whatever you want, and you'll add a policy. We're gonna give it view only policy. So you'll add your collections, give it read only access. That way it can't, like, update or delete or anything when it's in the MCP. If you want the MCP to do that, you can totally give it permissions to create, read, and update, and do whatever you want with it. But I recommend giving it, like, the lowest level of access at the start so that, you know, it's not gonna go in and break your whole setup here. Mhmm. So we've got our policy. I'm gonna create a new user too to grab a token from. So if we scroll down to our tokens, we're gonna generate it and save. And then we'll go back in here, and that's where you put in this bear here, authorization code. You put that token, click install, and that will install our direct as MCP encursors. So now we're ready to go for that when this is finished. So let's go check on it. So what we've done is create a role, and then the role has the token? Yes. So the the user is what who has the token. So this MCP user has the role that we created. I created it all within the same page, but this person has the role, and then they have the token. Mhmm. So if you go into our settings here and look at the MCP role, we've got our view only policy that we set up Mhmm. For our different collections. And the users in the role is that MCP guy. So now it looks like it did finish the content. If we go back into our content page, now you can see it's filled it in with our sample data. I gave it a few guidelines of the type of things I wanted it to have, but, otherwise, it just kinda came up with this on its own, sample content. So it's given us a starting point. So now that we've got some sample content, we are going to go back in to cursor. I'm gonna grab this really long prompt I've created telling it basically we are an AI agency. We're creating our website. I gave it I told it to use next. I also made sure to tell it to use the Directus MCP server. This might take a minute, so I'm gonna go ahead and get it started. We are in a completely empty folder here, so I'm gonna have it do all of the coding for me. Let it create what we need here. It's gonna make sure to look at our schema via the MCP. So we want it to not guess on what structure we have of our data. We want it to actually look at it. Like we said, we could create the data directly from Cursor via the MCP. You would have to make sure you give it, the right permissions. So that that might have to be an admin token in order to generate new schema with your MCP. So if you're comfortable doing that, you totally can. Interesting. So once that's done, we'll have something that's working like a front end running locally that's fully functional, pulling data from our deployed direct us instance. Correct? Yep. Okay. Yep. And any changes you make in direct us would be reflected in your local front end that you've got here. And then as the final step, we are going to do to deploy this front end to railway so that you have a full production application front end and back end. Yeah. I have cursor set up, so I could just automatically runs everything. I don't, like, manually approve stuff. I just I don't trust it. Yeah. I gotta make sure I know what it's doing. I've been burned before. I don't know. The the models are they keep getting better and it's like, I'm like, I would've done the same thing. So I don't know. I feel like it's just faster and you can just like, you know, sit back, relax, grab your favorite drink while it gets to work. You come back to it and you're like, woah, this thing actually works. So Yeah. I think I'm a bit of a control freak. I can't let it just just go. I'm telling you, like, once you let go and you see, like, it actually works and it's like, I think, I think you'll be plenty surprised. Then you're like, wow, that's actually a really fun. Cause like now, you know, if let's say for whatever reason, someone brings you on Slack, you can play for guys. Like, you come back to us, like, oh, I forgot to approve. Yeah. That it's done. Yeah. Yeah. I I I think, it's a much more fun experience when you just let it kinda do its own thing without you steering it. But I do understand the, appeal of, you know, wanting to see what it does. But also, like, we're starting from scratch, so there's little, like, room for error, I would say. But yeah. There's nothing for it to break. It's creating it from scratch. So Yeah. Yeah. We're pretty comfortable letting it just do what it wants to do here. So it looks like it actually has gotten the types if you look. So it pulled this from our MCP. It checked in, Directus, and it got our back end and what types we have there. So that's good. Looks like it's I did tell it also to create a little connection a net little connector to direct us here, a fetcher. So it looks like it's done that, getting different gets to get the data. And it will be able to pull in our deployed, like, like, URL and actually set it as an environment variable? Yes. It will. So I it might make me make the e n v, but I've got it. It's supposed to use the direct URL here, and it's supposed to use the token that we created. So, we'll see. I've done this a couple times just to see how it would do, and it it doesn't always know to grab the URL. If it's really smart, it can grab it from the cursor settings, and it can get the token and the URL, but it doesn't always manage that. You'll notice I'm in auto mode, so it's a different model each time, so it might not be a very smart model. But we at the end, we'll give it the proper e n v variables, and it'll be able to connect to our deployed instance. Excellent. Yeah. Honestly, again, I I like my to choose my own models, You should use, like, a, you know, g p three five point four or, like, an Opus 4.6 and because I don't know. I don't like that, I would say I wouldn't trust the auto. Like, which one would affect? Like, I wanna be I want a consistent experience. So then if, like, anything starts feeling weird, then I can blame the model. I was like, oh, you know, like, Opus is acting really dumb lately, that sort of thing. So It it does just feel like from one day to the next, though, even if you have a model that you know is good, it it doesn't always behave exactly the same. Yeah. Sometimes it feels like, you know, you're you're just rolling the dice. Sometimes it's like, woah. This is it. Like, this is the next big thing. I was like cause like, sometimes I, I, I asked it a question and I wanted it to check against like, you know, our internal, GitHub repo. And it's like, Hey, do we support this thing? It was like, yes, you do. And I told her like, oh, show me the code. It's like, after checking, apparently, this is not supported. I was like, come. Yeah. And this is like the smart and I told it, like, oh, you know, UltraThink, you know, work really hard, but it's like, I guess sometimes it's just, you know, not that type of day. But I think here it's like it's it's it's working well, based on what I'm saying. Yeah. It looks like it's getting something. Yeah. It's messing around with the configs now. Interesting. What is what's it doing to the Next. Js config if we open it up? Just out of curiosity. Technically, you know, we're we're not supposed to look at the code, you know, and, because, you know, I was just shipping it, but I'm also curious. Okay. It's setting, like, the assets URL, I assume, for, like, files and stuff. That's pretty awesome. Yep. And while it does that, I'm actually gonna grab my last prompt, which is a prompt to have it, push to GitHub so that we can get it ready for deploying the front end. I'm gonna grab that. Go back in here. I think it's almost done. It feels like it's it's doing the read me. So Yeah. Probably at the end. But yeah. I would say, there is actually a like, you you can definitely, you know, start with this workflow. It's totally fine. But if you have, like, the AOS CLI setup, it could just deploy it to your existing project. And this way you don't even have to worry about git being installed, get, having a GitHub repo or anything. But when you have it with git, well, the benefit is, anytime you make, changes through the front end, it will just automatically deploy. So it's like it's it's all trade offs based on, you know, how, structured and organized you wanna be. But yeah. Yeah. That makes sense. Alright. We did not create our E and V, so we're gonna create that real quick. And I would say that's the responsible thing for it to do. Yes. I don't didn't really like that it created its own ENV and, like, grabbed my things from the MCP. That was a little, uncomfortable. Yeah. Yeah. It then gets better this way. Yes. Look at our URL. Alright. And then we're gonna run it locally just to see it working. What does the static token do? So that's the one we created with for the MCP. We're using it here too. Okay. So that is the permissions that we allow in Directus. So if you give it, like, no read permission, it won't be able to see the content. If you give it access to read, like what we did, we gave it access to read all of these collections, then it can read the data. So that's where, like, you probably want a separate one from your MCP. We're just in the interest of time having it Yeah. One that is view all for all of our collections because we don't need to make changes with the MCP. But you probably would have permission to update and delete with your MCP as well. So you probably want wanted a separate one for your front end. So you'd probably wanna create a user role for your front end that gives it different permissions. Otherwise, you can have it set up to use your public permissions, which controls what the API data is, available without authenticating. But it's safer if you do give it a token so that your front end is only able to grab what you want, not, like, everything. So let's see if it worked. Moment of truth. Okay. There it is. Looks like some images are broken, but we've got a full website here. You can navigate around. It's got some prompts. It's got a button to get in touch. We've got a full website here. Obviously, you'd wanna iterate on it, fix the images, and, do a little honestly, it looks great. It doesn't need too much design or anything even. So that's a good starting point. We are going to go ahead and say good enough. The images can get fixed later. We are going to deploy this. And, of course, I copied other things, so I need to go back and get my prompt. So like you said, there's other ways to do this, but we are gonna have it pushed to a GitHub repo. We're pretending we have other people who wanna manage this code, and we want it in some place that we would have access to it as a team. So we've have a GitHub repo we've already set up. It's empty right now, and we're gonna push it into here. So I'm having it create a readme that'll tell users, our other team members, basically, is in our example here. We're deploying to Railway. Here's what you'll need. It'll mention the EMV variables we created here, because we're gonna need those. I'm actually gonna copy them now, so I have them ready. And then it it's gonna make sure everything's building properly with the prompt I gave it, and then it will push to GitHub for us. Yeah. And I noticed it said, like, even if you don't have, like, the environment variable set up initially, like, it should still deploy. It should still work. But once we actually set the these values, we'll be able to actually fetch the thing and have the same result we have locally, but actually have it live and deployed on a URL. Yep. It's updating the ReadMe. Yep. There's our ReadMe. I remember the times where I had you know, I used to write stuff by hand. Yeah. Now I just be like, you know, just instructions, just lines, like update, read me, and that's it. You know? Just just figure it out the rest. Alright. I think it's gonna commit. Yeah. I think it's already I believe I ran already a built. So, yeah, we see, like, the dot next directory. Maybe the build was under there. Yeah. It's running get the commands. Okay. I think it has committed everything here. So now What did I do? I think it's just the wrong link. Yeah. It did. Okay. So we are gonna now go in here. We're gonna add a service. Yep. GitHub repository. And what did I call it? Was it like AI agents or something like that? AI agents. Not sure. Oh, boy. Hold on. I pasted it down here. You should be able to see, like, all repos, but maybe you have too many. So, like I have so many. I really need to clean it up. Yeah. Yeah. That's me. Okay. Hold on. I'm gonna Give me to refresh. Yeah. Yeah. Just don't wanna have my mega list of repos up on the screen so one's. But I think you might need to refresh the, so I tried to push the code. Right? Oh, it didn't. Yeah. I don't know if it pushed. I mean, I guess you need to stop. Yeah. See It's getting me with the permission things. You're right. I should just let it do whatever it wants. Yeah. But now it's I don't know if it's pushing it, and then it should be good. So now if you go to right way, I would say if, if you refresh, you should be able to see it. Yeah. We have code. The code is now here. Yep. Now when you add, we might need to like refer, I think it should just fetch, but you can try. Does it not like my Maybe just try refreshing, like, the, entire Page. Dashboard. Yeah. Yeah. Just in case. Yeah. Okay. And Okay. Let's try again. Interesting. I mean, if you want, what you could do is maybe it's just an issue with, like, get up not pulling. So, like, you can copy the full URL. And when you add, you can just paste that URL. Okay. The it looks like it found it like that. So let's Yeah. Yeah. Alright. So that is how you connect it, and then we're gonna just need to hit deploy. Yep. Yes. But I wanna put it up by the other ones first. Yeah. And that was in there. Yes. So I wanna also add our e and v variables before we get started. So they're actually automatically inferred, like, at the bottom. Oh. You can see them. You were right. Suggest variables, and then you can update these values and click add, then you're good to go. There you go. So let me grab where I pasted them over here. You can also have, like, go the route you were going through, like the raw editor, copy and paste that other thing, but just just wanted to point it out. Because, like, to me, it's like, it's one of those things that when you don't know it, you're like, oh, that's neat. That's kinda how I always felt about it the first time I tried it. Yeah. You're right. That is cool that it already kind of pulled up what we needed. But, yeah, you could also open raw editor and just paste in the whole EMV file that we had ready. Yep. And then you can click add to add everything. Yep. So these this is the token we have in our local one, and then this is the URL of the same thing in Railway. You guys can also do internal URLs. Right? Is that something we could use here if we wanted to use the local URL of this? So you can do, like, a dollar sign, two curly braces, and then you should get auto complete for, like, the URL. So, actually, you will be able to see a line where they're both connected if you wanna try it out. And then what will happen is, now whenever if the URL changes, it will just automatically update the other one as well and redeploy it. Okay. So that's like using the internal URL reference variable. So I I believe you the way it would work is you just do, so the on directives and the variables themselves, like, if you go to variables. I believe what's the yeah. I guess it's public URL is the value that we will want. So wait. If you go now to the AI agency service and then go to the next public directors URL, because that's the URL value you wanna set. Right? So that more menu and then edit. And then if you do, dollar sign and then double curly braces, you'll be able to actually pull in so I believe the name of the service is directed. So you just do direct us, and then it's gonna be dot yep. Do you see? And then you just choose the public URL Yep. From this list. Got it. So now when you choose it, it's just gonna automatically give you the right URL and it will just be able to read it. So if the other one updates, you just do it. But for this, you're gonna need to hit redeploy again. So it will take some time. So I'd say we can do this after you can cancel. But yeah. And I would say there's actually another cool feature because might as well, if you go to settings, like, okay. It's done. So we can actually test the thing. But we can update, like, the domain to, like, a actual domain because you can now buy domains on railway. Nice. Okay. Yep. So this will give to you just a domain for, the front end itself, and it will have, like, a .up.railway.app at the end. You'll need to specify a port, which I'm not sure what it is on. I I think it should be 3,000. Like, 3,000 should just work. Yeah. Yeah. And if we visit it, might be the incorrect port. You could try eighty eighty as well because that's what I was suggesting in the beginning. Yeah. There we go. There we go. So there is our front end deployed. Yep. And we have a live app. Yeah. Awesome. So that was pretty fast. We went from nothing to a fully deployed front and back end here. Let's see. Was there anything else we should know about Railway and getting this set up here? I mean, if you hit deploy now, it will just redeploy, and you'll actually be able to see on the canvas. There's, like, an arrow, and this, service, like our front end is, referencing the direct us. And that's how you have, like, a connection between the two. And, yeah, other than that, it's like, you can just have everything in one place. Again, like, if you go now to the AI agency service and you click on settings and you also go to networking, from the right. Yeah. And then you can actually choose a custom domain. So when you click on it, you should actually be able to buy a domain straight from here. So like, if you come up with that domain, you can just do it from here as well. Or if you go to like ferry.com\u002Fdomains, you can do it. But yeah, I would say that's kinda like you now have fully like everything in one place, but you know, took us not a lot of time, and we have, like, a fully, fully deployed working back end, front end. Just, yeah, everything is in one place. Yeah. So why don't I recap what we did? We started with a simple idea launching a fictional AI automation agency. We launched infrastructure on Railway. We generated a structured back end with Directus. We populated it with fake content. We generated the front end using Cursor. We deployed the whole thing back to Railway. That's the build it part of today's theme. The key takeaway is that when interfaces can be generated quickly, the back end data model becomes the strategic layer. If someone watching wants to try this themselves, what's just the easiest way to get started on the railway side of things? The easiest way, I would say going to the template you shared would be the easiest place. And if they, you know, wanna use direct this. And also, by the way, I forgot to mention. So if you actually, like, close the, service here, there's like an agent in the top right corner as well, where you can actually also ask questions about your stuff, like actually ask it to make changes and it should be able to make changes, directly. So I believe if you just say like, oh, update, make this update, it should figure things out on its own. Awesome. So you guys got some AI in here too? Yeah. Yeah. In case someone wants everything in the dashboard, they can, but also if they want, the, to use whatever their coding agent of choice, they should be able to do it. But yeah. Cool. Well, thank you for joining us. I hope everybody has a good rest of your Leap Week. Sounds great. Thank you so much for having me. Yeah. Thank you for joining us.","published",[18,25],{"people_id":19},{"id":20,"first_name":21,"last_name":22,"avatar":23,"bio":24,"links":10},"5b04e96a-3249-41b4-a7ad-eccde77afef2","Lindsey","Zylstra","d0607d16-c28d-4fd4-8930-68f7b2277746","Marketing Engineer at Directus",{"people_id":26},{"id":27,"first_name":28,"last_name":29,"avatar":30,"bio":31,"links":10},"98307874-2b80-4796-a5b3-393a090605e9","Mahmoud","Abdelwahab","c34c3525-10a6-4e1e-9b94-3fcc2079484e","Senior DevRel Engineer at Railway",[],{"id":34,"number":11,"year":35,"episodes":36,"show":44},"289f6534-7fdd-46df-8c00-89a75469fe41","2026",[37,38,39,4,40,41,42,43],"bbaa3063-6fbe-4d96-bbc7-e50672f9a308","f885409e-0ace-41e5-aca3-faf4dcd7659b","7271f0be-33fd-4cea-b7bd-9c63e74969e1","a10f99c3-6b45-46e6-b703-64366f150c57","1310befc-e361-4e19-848f-d685c19dddef","37e28ea2-bec3-40bd-8b09-b9fbb47c1759","68536266-9502-4df8-a295-ef082dfe6fd0",{"title":45,"tile":46},"Leap Week","62816023-fa7e-4a76-b9a1-2733ee2093a6",{"title":10,"meta_description":10},{"id":40,"slug":49,"season":34,"vimeo_id":50,"description":51,"tile":52,"length":53,"resources":10,"people":10,"episode_number":54,"published":12,"title":55,"video_transcript_html":56,"video_transcript_text":57,"content":10,"seo":58,"status":16,"episode_people":59,"recommendations":62},"web-to-backend","1176299027","The web is full of data. We'll show you how to extract it for AI, automation, or whatever you're building.","d8202d8b-9701-4cf8-bee1-b4f595512685",43,5,"Web to Backend","\u003Cp>Speaker 0: Alright, folks. Welcome to another session here at Directus Leap Week. I am joined by Leo Gregorio from Fire Crawl. We've got some exciting stuff for you today. We're gonna be taming unstructured data.\u003C\u002Fp>\u003Cp>Right? Turning the wild web into ready APIs using FireCrawl and Directus, to to serve those APIs. I am Brian Gillespie, a staff product engineer here at Directus. And, again, joined by special guest, Leo Gregorio from FireCrawl. Leo, maybe maybe I'll kick it over to you for a quick intro for everyone.\u003C\u002Fp>\u003Cp>Speaker 1: It's a pleasure to be here, Brian. I've used Directus for quite a long time, and I'm stoked to be here.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. I'm super excited to have you, man. Yeah. I've messed around with FireCrawl a little bit before this. I agree.\u003C\u002Fp>\u003Cp>I'm I'm no way an expert. I'm glad to hear that you've messed with Directus before. But for the folks who are new to FireCrawl, you know, can you take a few moments just to explain what it what the tool is? Like, what do you guys do? What do you enable for developers?\u003C\u002Fp>\u003Cp>Speaker 1: Sure. Over at FireCrawl, we specialize at providing web data to agents. Not only agents, we can also have endpoints where you can build your data, data web fetcher upon those APIs. But we mainly focus on providing data over to agents in a way that, it's it's basically giving years' eyes over to your cloud code instance to wherever you're really building, to fetch for data. You should be thinking of FireCrawl to do that.\u003C\u002Fp>\u003Cp>Speaker 0: Sick. What are, like, some of the, like, specifics within that? Like, what what capabilities does FireCrawl have?\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. To fetch for data, I like to compare with our manual way of fetching data. So if you want to fetch a specific website, what you would do is search for something. So we have that specific endpoint on searching. You can then use scrape to scrape that specific URL.\u003C\u002Fp>\u003Cp>And from the scrape endpoint, we have a bunch of different formats that we will likely be exploring through this presentation, which are screenshot. You can actually screenshot the entire page. There's a bunch of different ways to take that screenshot. It can be exactly what we're seeing right now. It can be of the entire page, mobile.\u003C\u002Fp>\u003Cp>So that's just an example of one of our formats. We can fetch for summary. We allow the agents to execute and interact with the page, and that helps a lot with context, for example. So you're not only fetching pure HTML. You can fetch clean markdown.\u003C\u002Fp>\u003Cp>You can fetch exactly what you need from that given page, and that's that's the beauty of it. That's what we try to optimize to. And, yeah, we we've been been pretty successful, while doing so.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah, man. I I've watched the growth of the last couple years, so kudos on that. It's it's a really interesting tool. And as I understand it, like, you guys have an agent thing on your endpoint as well. So you could you could basically call an agent your agent from a different agent.\u003C\u002Fp>\u003Cp>Yes. And and get really Yes. Metal with it, which is nice. I I love the ability to, again, just throw in a URL in here and get structured data out of the other side because it Oh, yeah. Complements perfectly with Directus.\u003C\u002Fp>\u003Cp>And what we do is, like, if if you give us structured data, we'll give you ready to go APIs and permissions and and all of that. So, super excited for this. Alright. Let's get down to brass tacks. Right?\u003C\u002Fp>\u003Cp>What are we gonna build today? So the scenario that I've got, basically, I've got a directory of different software. This is on my own site. So this is a a Better Sign Shop. I love the print and sign industry.\u003C\u002Fp>\u003Cp>I can't get out of it even though I've tried. Software comes up a lot. You know, I I guess it does in any industry these days. But, for sign and print, you know, how do they manage their business? There's a bunch of different tools for that.\u003C\u002Fp>\u003Cp>And we've got a directory here that, is sorely unmaintained. Right? All this data was collected manually some years ago, and we've got things like features of a particular software. On on some pages, we have pricing information. But, you know, it's a lot to manage all this personally.\u003C\u002Fp>\u003Cp>It it would be a lot to put on the vendor of the software to maintain. It'd be a lot for anybody on my side to say, okay. Keep all this up to date. Does this seem like a good use case for Firegirl?\u003C\u002Fp>\u003Cp>Speaker 1: It's a perfect use case. Since we provide structured data, you can fetch for that. There's plenty of ways we can fetch for that. So if you already have the specific URL, you can use scrape or you can, like, use the agent that you mentioned. And by the way, the design of that website is pretty neat.\u003C\u002Fp>\u003Cp>Okay. Thank you, man. Yeah. If my, pretty neat.\u003C\u002Fp>\u003Cp>Speaker 0: Okay. Thank you, man. Yeah. My, I I've got three daughters. They love pink and purple.\u003C\u002Fp>\u003Cp>So, hey, like, anything that I do personally, like, it has to have pink in it. Right? Alright. So let's let's talk through maybe how we're gonna build this real quick, and then we will dive into actually, like, going through it. Right?\u003C\u002Fp>\u003Cp>So we've got Okay. We've got a software collection. You know, mostly, I wanna have, like, pricing data. We've got, like, a name. Let's just call it, like, a summary.\u003C\u002Fp>\u003Cp>We're providing a URL. What else do we have? We got pricing data has to have a specific format, which should be good. I'm yeah. Maybe pricing is the the best place to start.\u003C\u002Fp>\u003Cp>So a like, that is going to be I don't know the pricing URLs for this, but I do have the domain. So, like, walk me through what what this is gonna look\u003C\u002Fp>\u003Cp>Speaker 1: like on the FireCrawl side. So from your website, it seems like you already have specific URLs. It doesn't seem like the idea is having a bunch of different URLs. It seems like it's focused on quality. Mhmm.\u003C\u002Fp>\u003Cp>Therefore, if we were to have the specific pricing page, it would be better just because we can send off just one endpoint. It would be just scrape. So if we have that in the database, we can just search for that specific URL and get everything, all the structure, the price, we can check track if the the price changed, everything. If we don't, if the idea is scaling this, so for example, if your website were to have anyone add, any, type of software there Mhmm. This would scale up.\u003C\u002Fp>\u003Cp>And we wouldn't have control over what is the URL paid, the the pricing page, the URL for everything. Right. And then we'd have to create some kind of system where we search and then actually find that that the the pricing page. We can either do that with crawl, because inside of crawl, we have an LLM, or we can use agents to grab everything.\u003C\u002Fp>\u003Cp>Speaker 0: I I got you. What do you recommend in this situation? I'll just go back to FireCrawl here and go to my dashboard.\u003C\u002Fp>\u003Cp>Speaker 1: Because we I'd recommend going for for the pricing page, and we can test that inside of the playground for fire crawl. Let me send the link over.\u003C\u002Fp>\u003Cp>Speaker 0: I will look here in the chat. Here it is. Alright.\u003C\u002Fp>\u003Cp>Speaker 1: So grab any pricing page that we already have for one of those software queries.\u003C\u002Fp>\u003Cp>Speaker 0: Aware of. Okay. I think I think this is the right one. Let me make sure. Pricing.\u003C\u002Fp>\u003Cp>Speaker 1: Okay.\u003C\u002Fp>\u003Cp>Speaker 0: Yep. There we go.\u003C\u002Fp>\u003Cp>Speaker 1: Nice. So\u003C\u002Fp>\u003Cp>Speaker 0: we're just gonna throw this\u003C\u002Fp>\u003Cp>Speaker 1: in. Perfect. As for the format, you can select, let's see. Jason.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Let me zoom back out just a little bit. Alright. So we're gonna\u003C\u002Fp>\u003Cp>Speaker 1: hit edit options. Perfect. And that's basically an LLM that understands what it's seeing inside of the page, and you can prompt it to fetch only the pricing data from that specific page.\u003C\u002Fp>\u003Cp>Speaker 0: Gotcha. Okay. So let me pull this up over here. Oh, format JSON. Alright.\u003C\u002Fp>\u003Cp>So we're gonna edit our options. Right? And here is just basically going to be what I want from the pricing data. Right? So we are going to find this particular one in here.\u003C\u002Fp>\u003Cp>We're gonna look at this and alright. So Perfect.\u003C\u002Fp>\u003Cp>Speaker 1: You already have the schema. Like a\u003C\u002Fp>\u003Cp>Speaker 0: Like a can I just paste JSON here, and it'll pick this up Yep?\u003C\u002Fp>\u003Cp>Speaker 1: Or no? I don't think you can, that would be all It's gotta be, it's gotta be\u003C\u002Fp>\u003Cp>Speaker 0: JSON schema. Okay.\u003C\u002Fp>\u003Cp>Speaker 1: But that's a nice idea. We should add a way to get grab an example and produce the schema. That's that's something.\u003C\u002Fp>\u003Cp>Speaker 0: Starting price, that's gonna be a number. Actually, it's a string.\u003C\u002Fp>\u003Cp>Speaker 1: I I think the first one should be plans, and and it should be an array. Okay.\u003C\u002Fp>\u003Cp>Speaker 0: It should be right. Nested dot. Yeah. Got you.\u003C\u002Fp>\u003Cp>Speaker 1: And then it's an array of objects.\u003C\u002Fp>\u003Cp>Speaker 0: Objects. Perfect. Starting price. And we want pricing\u003C\u002Fp>\u003Cp>Speaker 1: period. Oh, no. For each Actually, inside of the the array of objects, it would be, like, the the plants. Oh, yeah. I think you I think you were go going right, and I yeah.\u003C\u002Fp>\u003Cp>We reversed it. I was just, like, the array.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Sorry. No. All good. I I started typing it, and I realized it as well, man.\u003C\u002Fp>\u003Cp>No worries. It's usually me, like, fat fingering typos on these things instead of, like, actually typing something completely wrong. But, billing cycle. Cool. Billing period.\u003C\u002Fp>\u003Cp>And then we have features, which is gonna be an array itself, array of strings. I got highlight, but I don't think we really need highlight in this case. Okay. Hit save options. Okay.\u003C\u002Fp>\u003Cp>Yeah. Cool. So anything else that I need to That's a checkbox.\u003C\u002Fp>\u003Cp>Speaker 1: It. We can hit start scraping, and let's see what we get. Depending on what we get, maybe the website has a heavy JavaScript, and that might require us to wait a couple seconds before applying the scrape. So that's where the other options comes in. Sick.\u003C\u002Fp>\u003Cp>Alright. There it is.\u003C\u002Fp>\u003Cp>Speaker 0: Price. Sick. Okay. Shopbox Express. Features.\u003C\u002Fp>\u003Cp>Yeah. So we could just it's probably wise to pull this up side by side just to verify what's what's coming in. One zero nine plus 29 user a month. The billing cycle is monthly. We have features, pricing tools.\u003C\u002Fp>\u003Cp>Yeah. This is this is pretty much one to one. I like that the thing truncated, like, some of this data because it's not really necessary.\u003C\u002Fp>\u003Cp>Speaker 1: The show can specify a bit more because the pricing, depending on the the the the column, maybe we don't want exactly the same way the the pricing is there. But\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. And is that done just via the prompt or, like, in the actual JSON structure? Or\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. We we can decide either if, because we can have, like, a strategy where the price could be a number. Right? And then it would be forced to place only a single price and then have, like, optional prices, right Yeah. I get it.\u003C\u002Fp>\u003Cp>Yeah. So it's really it's really up to whatever we're really building with that.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. So it looks like the, like, the the way I've got the front end for this particular thing structured. Right? We're relying on let's take a look at what the pricing plans look like. Right?\u003C\u002Fp>\u003Cp>So we've we've got this really specific, like, format here of, like, we need a number there that's large. Then we've got, like, some additional stuff, and then we have, like, the month. So\u003C\u002Fp>\u003Cp>Speaker 1: let let's have a price, which is a number, a float. Right? And then, additional pricing information. How about that?\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. So we got price. That's a number. Alright. Let's just go back to direct us.\u003C\u002Fp>\u003Cp>Cool. Price. Alright. So would we just specify this, like, up in the prompt of, like, billing cycle should be something like slash month or slash is this the best way to inform it?\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. That that's that's one way that should work pretty fine.\u003C\u002Fp>\u003Cp>Speaker 0: Okay. In this case, we'll say billing period just captures additional per user or extra cost. Alright. Let's see what that does. Let's see.\u003C\u002Fp>\u003Cp>That gives us what we're looking for. And if not, we'll just work on actually integrating this into Directus instead of actually pulling. K. 29. Okay.\u003C\u002Fp>\u003Cp>Price slash per month. 29 per user per month. Yeah. So this is already looking much better. Sick.\u003C\u002Fp>\u003Cp>Yeah.\u003C\u002Fp>\u003Cp>Speaker 1: Okay. I think that's that's better for the front end.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. So if I hit go get code here, this is gonna give me what I need to integrate this inside Directus.\u003C\u002Fp>\u003Cp>Speaker 1: Exactly. You can get a JavaScript code on on on on the right.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Okay. Yeah. And I could also, like, get curls. There's a couple different ways to do this.\u003C\u002Fp>\u003Cp>Like, we could easily, like, write an extension inside Directus that could could do this for us. Since this seems pretty lightweight, we could just use direct as flows for this as well. And I I think we've even got, like, a fire crawl extension in the marketplace for this, but let's just let's just figure out how to do this, through flows. That might be the the easiest, quickest way here so we don't have to worry about distributing an extension anywhere. Alright.\u003C\u002Fp>\u003Cp>So Let's go. We will set up a new flow. I think we probably just, like, manually trigger this, and then we could set it up on a later. Alright. So we're gonna say fetch pricing data.\u003C\u002Fp>\u003Cp>Sick. Alright. We're gonna manually trigger this as a hook. We are looking for this is always whenever you do a demo, you always find flaws in your own software. We'd need a we'd need a search for this for sure.\u003C\u002Fp>\u003Cp>Alright. So we're gonna manually trigger this on the software collection. Next, we would just do what we call a HTTP webhook requesting URL. That's the operation. We'll say call fire crawl.\u003C\u002Fp>\u003Cp>And I'm gonna definitely have to roll the token after this. But\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. I was just about to mention.\u003C\u002Fp>\u003Cp>Speaker 0: Luckily, this is also a local host as well. Alright. So we're just gonna add a header, authorization. Bear no. I could also, like, throw this in the EMV, but we're not gonna bother with that for now.\u003C\u002Fp>\u003Cp>Authorization. There's those typos that I spoke about. What else we need? Content type. Application.\u003C\u002Fp>\u003Cp>JSON. Sick. Okay. And then here's the data. So that's what we're sending in the body, except the only thing that we're gonna have to change here, right, is the URL that we're we're triggering this from.\u003C\u002Fp>\u003Cp>So Exactly. In this case, Directus has this functionality where we could do something like this, where we add, like, a a mustache syntax, and we can say trigger dot body dot, I don't know what we're gonna call this pricing URL. I I think it may be payload. I don't know. It will we'll test this in a moment.\u003C\u002Fp>\u003Cp>So, basically, what this will do is populate that from what we call the the trigger. And I'm just gonna go in and because we're not tracking the URL, maybe we just we do that pricing URL right here. Pricing page URL stream. Cool. Hit save.\u003C\u002Fp>\u003Cp>Alright. And before we add anything else, alright, all I'm gonna do is go in. We're gonna test this out. Fetch pricing data. Excited for this.\u003C\u002Fp>\u003Cp>Why didn't it show? It's probably fetching data, but it is not did I forget to save it? Requires selection. Oh, I guess I did forget to save it. Pricing URL.\u003C\u002Fp>\u003Cp>That happens. Alright. Sick. There we go. Got an input.\u003C\u002Fp>\u003Cp>Save. Okay. Now alright. This failed anyway. Right?\u003C\u002Fp>\u003Cp>Bad request. So I had to look at that. Let's see. But now if we go into ShopVox, here we go. It fetch pricing data.\u003C\u002Fp>\u003Cp>We've got the URL somewhere. Run flow. Cool. That triggered successfully. I can just go back to the flow.\u003C\u002Fp>\u003Cp>Normally, I would have these, like, set up differently. But alright. So we've got the body. No. It's not under payload.\u003C\u002Fp>\u003Cp>That's why it failed. Okay. So here's the pricing URL. It's triggered. So, basically, direct as flow's easy way to create these automations.\u003C\u002Fp>\u003Cp>Whenever you run a flow, each op each step or operations as we call them, they append their data, to an object that you could pull from. So in this case, the trigger has a special one where it has a dollar sign in front of it, and it's gonna be trigger.body or, well, it should be payload.body, I thought. Pricing URL. It should be that data. Here's what we passed in, but, yeah, we could see it's it's undefined right here.\u003C\u002Fp>\u003Cp>So that's probably where the issue lies. Trigger that body.\u003C\u002Fp>\u003Cp>Speaker 1: A column? An additional column at the end, maybe?\u003C\u002Fp>\u003Cp>Speaker 0: Trigger that body.\u003C\u002Fp>\u003Cp>Speaker 1: Oh, no.\u003C\u002Fp>\u003Cp>Speaker 0: No. No. This should be trigger.payload. Here's the options passed. Trigger.payload.body.pricing URL.\u003C\u002Fp>\u003Cp>That should be it. Let's just add what we can do is add an intermediate step in here. I'm not sure why this is not populating correctly. Triggered. Let's try I just wanna see what payload comes back with.\u003C\u002Fp>\u003Cp>Do you have to pass HTTPS for your for the API, or will it\u003C\u002Fp>\u003Cp>Speaker 1: pick up without HTTPS? It it should be able to pick up without HTTPS.\u003C\u002Fp>\u003Cp>Speaker 0: Okay.\u003C\u002Fp>\u003Cp>Speaker 1: It should just add it there.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. Let's see. Still showing undefined. Trigger. Why are we not getting this information that we need?\u003C\u002Fp>\u003Cp>Alright. Let's put in an intermediate step here. Just gonna basically run some JavaScript. We have a way to do that. We're just gonna use run script.\u003C\u002Fp>\u003Cp>I'm just gonna call this format. And here, what it does, it it receives all the data up to this operation that's already ran. And I just wanna return data dot trigger. And I'm I'm just gonna see I'm gonna wire this up, make sure we're getting that data properly. Flows.\u003C\u002Fp>\u003Cp>Dropbox. HTTPS. Run flow. Fetch our pricing data. Here we go.\u003C\u002Fp>\u003Cp>Unexpected token. What are we even doing here? The demos. The the demo gremlins. Unexpected dot.\u003C\u002Fp>\u003Cp>Why is it not liking that? Oh, yeah. Duh. Return data. Let's do it this way.\u003C\u002Fp>\u003Cp>Alright. Now open a new window. Get a little smarter here. Close. Fetch pricing data.\u003C\u002Fp>\u003Cp>URL. K. Refresh. What do we get? Okay.\u003C\u002Fp>\u003Cp>Cool. There we go. We can see the data. I don't know, again, why it wasn't liking that, but we'll just wire this up now. Let's see if this actually works.\u003C\u002Fp>\u003Cp>We're gonna call formats. And, basically, in this case, let's just return dot payload body. Alright. We can see we're getting oh, okay. I don't see a payload.\u003C\u002Fp>\u003Cp>That's probably where we were going wrong. Should just be body. That's my fault. Alright. So let's test one more time.\u003C\u002Fp>\u003Cp>Test. Here we go. Okay. There we go. Data.\u003C\u002Fp>\u003Cp>Alright.\u003C\u002Fp>\u003Cp>Speaker 1: There it is.\u003C\u002Fp>\u003Cp>Speaker 0: This is gonna this is gonna work this time. 100%. Here we go. Let's go. We are going to actually now we're gonna fetch that from format.payload.\u003C\u002Fp>\u003Cp>Actually, it's not, is it? Body.pricing URL. Nope. It's just nobody. There we go.\u003C\u002Fp>\u003Cp>Format .pricing URL should be good Or did I data dot pricing URL. Gotta help us. Format dot data. Alright. It is going to work this time.\u003C\u002Fp>\u003Cp>We are going to alright. So now we could see. Right? This is, we could we could actually set this up to be asynchronous as we wanted, but it is waiting for that to come back. And now bada bing bada boom.\u003C\u002Fp>\u003Cp>What do we have? This is the this is the data structure that we want. Right? Yeah. Exactly.\u003C\u002Fp>\u003Cp>Sick. Okay. Alright. So now we just need to basically wire that up to direct us. And in this case, it's going to be, let me make this full screen again.\u003C\u002Fp>\u003Cp>Speaker 1: And we'll Additionally, just to make this workflow, more reliable and just continue applying changes if something was actually changed from the pricing page, we can use the change tracking. So this would be just an additional format that that would instruct our workflow, yeah, something changed, right, instead of always fetching. Yeah. But that's Yeah. There's plenty more things that we we can we can add just to make sure everything's on point.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Okay. Well, it let me let's make sure we can update the software first, and then we will then we'll tackle that, man. Alright. So we've got software.\u003C\u002Fp>\u003Cp>That's our collection. We are gonna I'm just gonna give this full access for now. And what we need to do, this should work here. We're gonna do something like trigger dot keys dot zero. So it trigger dot keys.\u003C\u002Fp>\u003Cp>This is just an array of the the primary key for, whatever we're triggering on. I'll hit enter there. And then the payload that we're going to send I'm gonna save this real quick. We're gonna look at what we received back. So it's just Navigate the tree.\u003C\u002Fp>\u003Cp>Speaker 1: Dot JSON, basically.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. So it should be payload\u003C\u002Fp>\u003Cp>Speaker 1: Actually, dot dot data dot data dot data. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Dot data dot\u003C\u002Fp>\u003Cp>Speaker 1: Are there the q dot data there? Oh, oh,\u003C\u002Fp>\u003Cp>Speaker 0: I see JSON. Yeah. The it's like it's it's nested because the operation itself returns it in a data, and this is the fetch call that gets returned. So it's so it is data dot data dot JSON. JSON.\u003C\u002Fp>\u003Cp>Is what we want. Yep. Alright. So we should be able to do this where we have we're just gonna check the key here. I call this call fire crawl.\u003C\u002Fp>\u003Cp>I can always adjust these keys and that's, again, that's what it's gonna pin to. So it should be able to do something simple like this where we say firecrawl.data.data.json. And this is the pricing that we're going to update. I think it's surprising. What did we we'll open this up.\u003C\u002Fp>\u003Cp>What did I what did I call it here? Pricing data. Let me look at the actual data model real quick. Pricing data. That's that is indeed what it needs to be.\u003C\u002Fp>\u003Cp>Pricing underscore data. Alright. Now go back to the side by side, get that lovely shot of fire crawl. And now this is actually gonna work. Right?\u003C\u002Fp>\u003Cp>Yep. If the demo gods are on our side.\u003C\u002Fp>\u003Cp>Speaker 1: Always in the demo.\u003C\u002Fp>\u003Cp>Speaker 0: Here we go. Let's save this just just to make sure. Alright. Run flow. Now we should see this pricing change if everything works okay.\u003C\u002Fp>\u003Cp>Updated two fields. Something I'm assuming something failed. So it wasn't fire crawl. It was just me. BSS software.\u003C\u002Fp>\u003Cp>Keys undefined. Alright. Body dot keys. Okay. That's what it is.\u003C\u002Fp>\u003Cp>Operator error again.\u003C\u002Fp>\u003Cp>Speaker 1: Body dot keys.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. One more time. Fetch pricing data. Oh, no. You're gonna use all my FireCrawl credits before we even use them, actually.\u003C\u002Fp>\u003Cp>Yeah. Sick. Okay. Now we could see it actually worked this time. So amazing.\u003C\u002Fp>\u003Cp>Right? Now we've got this wired up where it will update the pricing data for us on on this schedule, or well, actually, not schedule. It's totally manual. Right? That's still not what we want.\u003C\u002Fp>\u003Cp>Talk to me about the the change tracking bids.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Change tracking is it's a format that you'll specify just like we specify JSON as a format, we can add chain tracking. And inside of that chain tracking, like we fetched to JSON, there'll be the chain tracking object that will tell us, hey. Something changed in the website or, no. Nothing changed.\u003C\u002Fp>\u003Cp>So you don't really need to update anything. And that's just that's an add on. We don't we don't really need to implement this in here. But it it's nice that you mentioned about wasting your Firepower credits because the reason why we went for scraping, which, by the way, we can use batch scraping to do everything, all at once, do everything in parallel. Every scrape you do to to every single page wastes one single credit.\u003C\u002Fp>\u003Cp>Gotcha. And the more you abstract Yeah. Yeah. Yeah. And the the the more you abstract, the more, like, for example, if you use crawl, crawl is a combination of map with scrape, then, you'll like, the credits will, be higher for that just because you're fetching the different pages.\u003C\u002Fp>\u003Cp>If you then abstract away to using agents, since the agent executes everything for us, you don't really need to think about searching, scraping, mapping, or anything. It should, it will be a bit more expensive just because it's handling a bunch of things that you really don't need to decide. So Gotcha. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: So it's like a sliding scale of what your what your your tolerance is of, like, hey. I know the exact workflow that I need. Exactly. You could you could be very surgical or, like, the agent Yes. Would just be, like, here's the URL.\u003C\u002Fp>\u003Cp>Here's some guidance. Here's the data that I want. Yes. Just work until you find that. Yes.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. And and we can get creative with that. So, while you were implementing, I thought we could use agents to fetch for the pricing page or for whatever pages from that specific software, and then just place that in over back in Directus, place it in the database, and then always have Fireflies scrape for that given URL. So we would just be using agent to populate the URLs that will be scraped, not needing to use agent every single time. So we can get really creative with this, to ensure that we're only scraping at the times that we really need to.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Okay. Sick. Let's let's see how we do that. Right?\u003C\u002Fp>\u003Cp>I'm in the the fire crawl dashboard here. Is this something that's better done here, or is it Yeah. Like, in\u003C\u002Fp>\u003Cp>Speaker 1: code or It's it's it's a nice place for us to test it.\u003C\u002Fp>\u003Cp>Speaker 0: Okay.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Alright. In there, you can actually just type in it could be a very high level. So example, just get the name of the you don't even need to place the URL, to be honest. You can just place Okay.\u003C\u002Fp>\u003Cp>Place in the name of the software and say, fetch the pricing for this software.\u003C\u002Fp>\u003Cp>Speaker 0: For let's see what we got. Let's do a different one. Let's say, core bridge here. First, the pricing for CoreBridge software.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: We'll just say sign industry in case they've got there's multiple. Right? So what's the the difference between Spark Mini and Spark Pro?\u003C\u002Fp>\u003Cp>Speaker 1: Spark Mini is so it's lightweight. It it's like the the comparison between using a Haiku model and a Claude or an op opus model. So one will be faster and the other will be just more efficient.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. Yeah. So here we are, Leo. We've seen, we've got this thing is it's fetching the pricing, and now it's asking for the details here. Is that, like, a failure for for me No.\u003C\u002Fp>\u003Cp>Speaker 1: It's actually instructing this? Yeah. It's a default just to fetch for more information. Usually, we don't give enough context to LLMs. So it's just asking for that context, but let's we can skip that.\u003C\u002Fp>\u003Cp>There's a button to just skip and immediately so down there Okay. Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: I see it.\u003C\u002Fp>\u003Cp>Speaker 1: We could just skip through it. It'll generate a specific schema that it should use. So it's already defining that schema. We always always try to output structured data. Yeah.\u003C\u002Fp>\u003Cp>Sick. And then now we and then Yes. Exactly. And then you can just run the agent, and it should fetch everything that we need. Since it it it runs asynchronously Uh-huh.\u003C\u002Fp>\u003Cp>You can ask for for it to alert us back whenever it's done.\u003C\u002Fp>\u003Cp>Speaker 0: So let's see what it's come up with. Right? It's got core bridge starter, SMB, core bridge plus. It's it's came back with, like, eight plans.\u003C\u002Fp>\u003Cp>Speaker 1: Oh, yeah. Let let's see if it's even, like, previous pricing for them.\u003C\u002Fp>\u003Cp>Speaker 0: No. It looks like there are that many plans. Annual. Okay. So it's divided into monthly and annual.\u003C\u002Fp>\u003Cp>Okay. Six. So, you know, I have $1.99, $3.99, or $1.29. I'm sorry. 549 and 899.\u003C\u002Fp>\u003Cp>Yeah. This is so the data checks out, man. I think, you know, that's a a concern with everybody these days, especially on, like, the the agents. Do you what's the the secret sauce? Is there yeah.\u003C\u002Fp>\u003Cp>Do you care to share? Like, how do you guys get accurate results on this stuff?\u003C\u002Fp>\u003Cp>Speaker 1: Oh, it's it's a mixture of system prompting using the best LLMs, understanding the flow of scraping. So Mhmm. Just having a fallback to when you need to interact with data, understanding, yeah, caching is is a part of it as well. So, yeah, it it's a bunch of I I I I feel like it's a bunch of little things that when summed up gets a pretty nice result like this.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Nice, man. Nice. Well, I know we're we're probably coming up on time. So, like, I won't go through the the trouble of integrating, like, the agent, but, you know, the fact that it would be, like, super simple here\u003C\u002Fp>\u003Cp>Speaker 1: Yeah.\u003C\u002Fp>\u003Cp>Speaker 0: Again, it's just a v two slash agent, and then we'll It would be\u003C\u002Fp>\u003Cp>Speaker 1: the same process as before.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Yeah. Actually, it may not be that bad. Alright. So we go back to our flow.\u003C\u002Fp>\u003Cp>You know, I would probably change this as well, right, where instead of a manual trigger, we could go into, like, a cron, and I could trigger this, like, once a week or once a month or something like that. We'll we'll just keep it manual for now. But let's let's try this out and see. So we're gonna change the\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. Change it over to agent.\u003C\u002Fp>\u003Cp>Speaker 0: And then it's just gonna\u003C\u002Fp>\u003Cp>Speaker 1: be need to add a prompt and a a schema?\u003C\u002Fp>\u003Cp>Speaker 0: Do we have the we got the prompt already? Right? And then the rest of this, we just\u003C\u002Fp>\u003Cp>Speaker 1: remove? Nope. Yeah. I I think\u003C\u002Fp>\u003Cp>Speaker 0: she can actually\u003C\u002Fp>\u003Cp>Speaker 1: remove everything since the prompt was specific to the URL it was given.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. We just need the the schema. Right? Alright. So we have prompts.\u003C\u002Fp>\u003Cp>Yep. Fetch the pricing for company name. Let's do the oh, I don't know how this is gonna work out. We'll see.\u003C\u002Fp>\u003Cp>Speaker 1: And we can actually explore even more. I I mean, we don't have enough enough time to do so, but every single schema from your website, not only pricing, but could be handled from the agent. So we could have many different arrays and and objects to explore all the sections of that.\u003C\u002Fp>\u003Cp>Speaker 0: Got you. Yeah. So it doesn't have to be, like, pricing. We could go in and basically, we're saying we could do the entirety of this and have Yes. Like, agent just continue.\u003C\u002Fp>\u003Cp>Yeah. Like like, attributes or any of that. Everything. Yep. Oh, man.\u003C\u002Fp>\u003Cp>That would be amazing. Exactly. Yeah. Sick. Okay.\u003C\u002Fp>\u003Cp>Well, hey. This is probably a good stopping point then. You know, on on fire crawl, man, like, you know, this is just as obviously, like, one simple workflow that we've touched on. Yeah. What's what's next for you guys?\u003C\u002Fp>\u003Cp>Or, like, what, is there anything you wanted to highlight out of the out of this?\u003C\u002Fp>\u003Cp>Speaker 1: Let's see. I believe I believe FireCrawl. I think from this simple demo, it shows implementing something as simple as this. You get you get a a lot. Right?\u003C\u002Fp>\u003Cp>So from just one endpoint or one tool from Firegral, you can explore, the entire web. And there is so many things that we can build with this. Because before before AI, before all this wave of AI, scraping was something really difficult. Scraping was something that, you'd have to handle everything. You'd have to try to treat and filter out specific words, specific sessions.\u003C\u002Fp>\u003Cp>And now with FireCrawl, you have an LLM handling everything for you. It just allows us to build a lot more. And not only to build with FireCrawl because we we showed an example where we're using the FireCrawl API inside of our app. Right? Yeah.\u003C\u002Fp>\u003Cp>But even coding, and I I'm seeing directors implementing more and more AI tools. Eventually We are then. Yeah. Eventually, you could integrate Firewall into those agents as well and have Firewall have information, have much more context because whoever uses AI, enough understands that context is king. Right?\u003C\u002Fp>\u003Cp>And if you parrot it with the best web scraper out there, then you'll you'll have a lot of, pretty much best results.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah, man.\u003C\u002Fp>\u003Cp>Speaker 1: But yeah. I mean Can we file or you guys\u003C\u002Fp>\u003Cp>Speaker 0: have got the CLI now as well. I see. Right? Yes. That could be the next step for me is, like, oh, let's mess with this.\u003C\u002Fp>\u003Cp>Right? Because they like, my day to day right now is a a lot of quad code certainly.\u003C\u002Fp>\u003Cp>Speaker 1: Yeah. And if the agent uses a CLI, it it can do a lot more. Right?\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Yeah. Sick. Okay. Yeah.\u003C\u002Fp>\u003Cp>So yeah. Another follow-up session then then for sure where we're gonna take the direct us MCP, and then we will add the fire crawl CLI, and then we will just turn cloud code loose. Because everything that we did here today, like, you could do that via, the direct SMTP. So I could have Claude build a flow that just calls fire crawl agent and, you know, we could simplify all this work. But Yeah.\u003C\u002Fp>\u003Cp>Speaker 1: It would be a matter of five minutes for it to build the entire page.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah, man. Sick. Sick. Cool. Well, Leo, man.\u003C\u002Fp>\u003Cp>And thanks for the the session. I I really appreciate you guys joining.\u003C\u002Fp>\u003Cp>Speaker 1: Thanks. Thanks for for having me. Just one more just an additional tip. Follow our YouTube channel.\u003C\u002Fp>\u003Cp>Speaker 0: Yeah. Our\u003C\u002Fp>\u003Cp>Speaker 1: YouTube channel, they're we're we're posting everything there. We launch a lot. So to keep up with the scraping industry, feel free to subscribe there.\u003C\u002Fp>\u003Cp>Speaker 0: Fire crawl YouTube. At fire crawl underscore dev. There you go. Yeah. Yeah.\u003C\u002Fp>\u003Cp>Sick. I see your face. There you are, man. Awesome, dude. Well, again, thanks for joining.\u003C\u002Fp>\u003Cp>This has been a fun session, and we could definitely follow-up again with a a run this back, man.\u003C\u002Fp>\u003Cp>Speaker 1: Absolutely. Alright. Excited to do it again.\u003C\u002Fp>\u003Cp>Speaker 0: Alright. See you. Thanks, everybody.\u003C\u002Fp>","Alright, folks. Welcome to another session here at Directus Leap Week. I am joined by Leo Gregorio from Fire Crawl. We've got some exciting stuff for you today. We're gonna be taming unstructured data. Right? Turning the wild web into ready APIs using FireCrawl and Directus, to to serve those APIs. I am Brian Gillespie, a staff product engineer here at Directus. And, again, joined by special guest, Leo Gregorio from FireCrawl. Leo, maybe maybe I'll kick it over to you for a quick intro for everyone. It's a pleasure to be here, Brian. I've used Directus for quite a long time, and I'm stoked to be here. Yeah. I'm super excited to have you, man. Yeah. I've messed around with FireCrawl a little bit before this. I agree. I'm I'm no way an expert. I'm glad to hear that you've messed with Directus before. But for the folks who are new to FireCrawl, you know, can you take a few moments just to explain what it what the tool is? Like, what do you guys do? What do you enable for developers? Sure. Over at FireCrawl, we specialize at providing web data to agents. Not only agents, we can also have endpoints where you can build your data, data web fetcher upon those APIs. But we mainly focus on providing data over to agents in a way that, it's it's basically giving years' eyes over to your cloud code instance to wherever you're really building, to fetch for data. You should be thinking of FireCrawl to do that. Sick. What are, like, some of the, like, specifics within that? Like, what what capabilities does FireCrawl have? Yeah. To fetch for data, I like to compare with our manual way of fetching data. So if you want to fetch a specific website, what you would do is search for something. So we have that specific endpoint on searching. You can then use scrape to scrape that specific URL. And from the scrape endpoint, we have a bunch of different formats that we will likely be exploring through this presentation, which are screenshot. You can actually screenshot the entire page. There's a bunch of different ways to take that screenshot. It can be exactly what we're seeing right now. It can be of the entire page, mobile. So that's just an example of one of our formats. We can fetch for summary. We allow the agents to execute and interact with the page, and that helps a lot with context, for example. So you're not only fetching pure HTML. You can fetch clean markdown. You can fetch exactly what you need from that given page, and that's that's the beauty of it. That's what we try to optimize to. And, yeah, we we've been been pretty successful, while doing so. Yeah, man. I I've watched the growth of the last couple years, so kudos on that. It's it's a really interesting tool. And as I understand it, like, you guys have an agent thing on your endpoint as well. So you could you could basically call an agent your agent from a different agent. Yes. And and get really Yes. Metal with it, which is nice. I I love the ability to, again, just throw in a URL in here and get structured data out of the other side because it Oh, yeah. Complements perfectly with Directus. And what we do is, like, if if you give us structured data, we'll give you ready to go APIs and permissions and and all of that. So, super excited for this. Alright. Let's get down to brass tacks. Right? What are we gonna build today? So the scenario that I've got, basically, I've got a directory of different software. This is on my own site. So this is a a Better Sign Shop. I love the print and sign industry. I can't get out of it even though I've tried. Software comes up a lot. You know, I I guess it does in any industry these days. But, for sign and print, you know, how do they manage their business? There's a bunch of different tools for that. And we've got a directory here that, is sorely unmaintained. Right? All this data was collected manually some years ago, and we've got things like features of a particular software. On on some pages, we have pricing information. But, you know, it's a lot to manage all this personally. It it would be a lot to put on the vendor of the software to maintain. It'd be a lot for anybody on my side to say, okay. Keep all this up to date. Does this seem like a good use case for Firegirl? It's a perfect use case. Since we provide structured data, you can fetch for that. There's plenty of ways we can fetch for that. So if you already have the specific URL, you can use scrape or you can, like, use the agent that you mentioned. And by the way, the design of that website is pretty neat. Okay. Thank you, man. Yeah. If my, pretty neat. Okay. Thank you, man. Yeah. My, I I've got three daughters. They love pink and purple. So, hey, like, anything that I do personally, like, it has to have pink in it. Right? Alright. So let's let's talk through maybe how we're gonna build this real quick, and then we will dive into actually, like, going through it. Right? So we've got Okay. We've got a software collection. You know, mostly, I wanna have, like, pricing data. We've got, like, a name. Let's just call it, like, a summary. We're providing a URL. What else do we have? We got pricing data has to have a specific format, which should be good. I'm yeah. Maybe pricing is the the best place to start. So a like, that is going to be I don't know the pricing URLs for this, but I do have the domain. So, like, walk me through what what this is gonna look like on the FireCrawl side. So from your website, it seems like you already have specific URLs. It doesn't seem like the idea is having a bunch of different URLs. It seems like it's focused on quality. Mhmm. Therefore, if we were to have the specific pricing page, it would be better just because we can send off just one endpoint. It would be just scrape. So if we have that in the database, we can just search for that specific URL and get everything, all the structure, the price, we can check track if the the price changed, everything. If we don't, if the idea is scaling this, so for example, if your website were to have anyone add, any, type of software there Mhmm. This would scale up. And we wouldn't have control over what is the URL paid, the the pricing page, the URL for everything. Right. And then we'd have to create some kind of system where we search and then actually find that that the the pricing page. We can either do that with crawl, because inside of crawl, we have an LLM, or we can use agents to grab everything. I I got you. What do you recommend in this situation? I'll just go back to FireCrawl here and go to my dashboard. Because we I'd recommend going for for the pricing page, and we can test that inside of the playground for fire crawl. Let me send the link over. I will look here in the chat. Here it is. Alright. So grab any pricing page that we already have for one of those software queries. Aware of. Okay. I think I think this is the right one. Let me make sure. Pricing. Okay. Yep. There we go. Nice. So we're just gonna throw this in. Perfect. As for the format, you can select, let's see. Jason. Yeah. Let me zoom back out just a little bit. Alright. So we're gonna hit edit options. Perfect. And that's basically an LLM that understands what it's seeing inside of the page, and you can prompt it to fetch only the pricing data from that specific page. Gotcha. Okay. So let me pull this up over here. Oh, format JSON. Alright. So we're gonna edit our options. Right? And here is just basically going to be what I want from the pricing data. Right? So we are going to find this particular one in here. We're gonna look at this and alright. So Perfect. You already have the schema. Like a Like a can I just paste JSON here, and it'll pick this up Yep? Or no? I don't think you can, that would be all It's gotta be, it's gotta be JSON schema. Okay. But that's a nice idea. We should add a way to get grab an example and produce the schema. That's that's something. Starting price, that's gonna be a number. Actually, it's a string. I I think the first one should be plans, and and it should be an array. Okay. It should be right. Nested dot. Yeah. Got you. And then it's an array of objects. Objects. Perfect. Starting price. And we want pricing period. Oh, no. For each Actually, inside of the the array of objects, it would be, like, the the plants. Oh, yeah. I think you I think you were go going right, and I yeah. We reversed it. I was just, like, the array. Yeah. Sorry. No. All good. I I started typing it, and I realized it as well, man. No worries. It's usually me, like, fat fingering typos on these things instead of, like, actually typing something completely wrong. But, billing cycle. Cool. Billing period. And then we have features, which is gonna be an array itself, array of strings. I got highlight, but I don't think we really need highlight in this case. Okay. Hit save options. Okay. Yeah. Cool. So anything else that I need to That's a checkbox. It. We can hit start scraping, and let's see what we get. Depending on what we get, maybe the website has a heavy JavaScript, and that might require us to wait a couple seconds before applying the scrape. So that's where the other options comes in. Sick. Alright. There it is. Price. Sick. Okay. Shopbox Express. Features. Yeah. So we could just it's probably wise to pull this up side by side just to verify what's what's coming in. One zero nine plus 29 user a month. The billing cycle is monthly. We have features, pricing tools. Yeah. This is this is pretty much one to one. I like that the thing truncated, like, some of this data because it's not really necessary. The show can specify a bit more because the pricing, depending on the the the the column, maybe we don't want exactly the same way the the pricing is there. But Yeah. And is that done just via the prompt or, like, in the actual JSON structure? Or Yeah. We we can decide either if, because we can have, like, a strategy where the price could be a number. Right? And then it would be forced to place only a single price and then have, like, optional prices, right Yeah. I get it. Yeah. So it's really it's really up to whatever we're really building with that. Yeah. So it looks like the, like, the the way I've got the front end for this particular thing structured. Right? We're relying on let's take a look at what the pricing plans look like. Right? So we've we've got this really specific, like, format here of, like, we need a number there that's large. Then we've got, like, some additional stuff, and then we have, like, the month. So let let's have a price, which is a number, a float. Right? And then, additional pricing information. How about that? Yeah. So we got price. That's a number. Alright. Let's just go back to direct us. Cool. Price. Alright. So would we just specify this, like, up in the prompt of, like, billing cycle should be something like slash month or slash is this the best way to inform it? Yeah. That that's that's one way that should work pretty fine. Okay. In this case, we'll say billing period just captures additional per user or extra cost. Alright. Let's see what that does. Let's see. That gives us what we're looking for. And if not, we'll just work on actually integrating this into Directus instead of actually pulling. K. 29. Okay. Price slash per month. 29 per user per month. Yeah. So this is already looking much better. Sick. Yeah. Okay. I think that's that's better for the front end. Alright. So if I hit go get code here, this is gonna give me what I need to integrate this inside Directus. Exactly. You can get a JavaScript code on on on on the right. Yeah. Okay. Yeah. And I could also, like, get curls. There's a couple different ways to do this. Like, we could easily, like, write an extension inside Directus that could could do this for us. Since this seems pretty lightweight, we could just use direct as flows for this as well. And I I think we've even got, like, a fire crawl extension in the marketplace for this, but let's just let's just figure out how to do this, through flows. That might be the the easiest, quickest way here so we don't have to worry about distributing an extension anywhere. Alright. So Let's go. We will set up a new flow. I think we probably just, like, manually trigger this, and then we could set it up on a later. Alright. So we're gonna say fetch pricing data. Sick. Alright. We're gonna manually trigger this as a hook. We are looking for this is always whenever you do a demo, you always find flaws in your own software. We'd need a we'd need a search for this for sure. Alright. So we're gonna manually trigger this on the software collection. Next, we would just do what we call a HTTP webhook requesting URL. That's the operation. We'll say call fire crawl. And I'm gonna definitely have to roll the token after this. But Yeah. I was just about to mention. Luckily, this is also a local host as well. Alright. So we're just gonna add a header, authorization. Bear no. I could also, like, throw this in the EMV, but we're not gonna bother with that for now. Authorization. There's those typos that I spoke about. What else we need? Content type. Application. JSON. Sick. Okay. And then here's the data. So that's what we're sending in the body, except the only thing that we're gonna have to change here, right, is the URL that we're we're triggering this from. So Exactly. In this case, Directus has this functionality where we could do something like this, where we add, like, a a mustache syntax, and we can say trigger dot body dot, I don't know what we're gonna call this pricing URL. I I think it may be payload. I don't know. It will we'll test this in a moment. So, basically, what this will do is populate that from what we call the the trigger. And I'm just gonna go in and because we're not tracking the URL, maybe we just we do that pricing URL right here. Pricing page URL stream. Cool. Hit save. Alright. And before we add anything else, alright, all I'm gonna do is go in. We're gonna test this out. Fetch pricing data. Excited for this. Why didn't it show? It's probably fetching data, but it is not did I forget to save it? Requires selection. Oh, I guess I did forget to save it. Pricing URL. That happens. Alright. Sick. There we go. Got an input. Save. Okay. Now alright. This failed anyway. Right? Bad request. So I had to look at that. Let's see. But now if we go into ShopVox, here we go. It fetch pricing data. We've got the URL somewhere. Run flow. Cool. That triggered successfully. I can just go back to the flow. Normally, I would have these, like, set up differently. But alright. So we've got the body. No. It's not under payload. That's why it failed. Okay. So here's the pricing URL. It's triggered. So, basically, direct as flow's easy way to create these automations. Whenever you run a flow, each op each step or operations as we call them, they append their data, to an object that you could pull from. So in this case, the trigger has a special one where it has a dollar sign in front of it, and it's gonna be trigger.body or, well, it should be payload.body, I thought. Pricing URL. It should be that data. Here's what we passed in, but, yeah, we could see it's it's undefined right here. So that's probably where the issue lies. Trigger that body. A column? An additional column at the end, maybe? Trigger that body. Oh, no. No. No. This should be trigger.payload. Here's the options passed. Trigger.payload.body.pricing URL. That should be it. Let's just add what we can do is add an intermediate step in here. I'm not sure why this is not populating correctly. Triggered. Let's try I just wanna see what payload comes back with. Do you have to pass HTTPS for your for the API, or will it pick up without HTTPS? It it should be able to pick up without HTTPS. Okay. It should just add it there. Alright. Let's see. Still showing undefined. Trigger. Why are we not getting this information that we need? Alright. Let's put in an intermediate step here. Just gonna basically run some JavaScript. We have a way to do that. We're just gonna use run script. I'm just gonna call this format. And here, what it does, it it receives all the data up to this operation that's already ran. And I just wanna return data dot trigger. And I'm I'm just gonna see I'm gonna wire this up, make sure we're getting that data properly. Flows. Dropbox. HTTPS. Run flow. Fetch our pricing data. Here we go. Unexpected token. What are we even doing here? The demos. The the demo gremlins. Unexpected dot. Why is it not liking that? Oh, yeah. Duh. Return data. Let's do it this way. Alright. Now open a new window. Get a little smarter here. Close. Fetch pricing data. URL. K. Refresh. What do we get? Okay. Cool. There we go. We can see the data. I don't know, again, why it wasn't liking that, but we'll just wire this up now. Let's see if this actually works. We're gonna call formats. And, basically, in this case, let's just return dot payload body. Alright. We can see we're getting oh, okay. I don't see a payload. That's probably where we were going wrong. Should just be body. That's my fault. Alright. So let's test one more time. Test. Here we go. Okay. There we go. Data. Alright. There it is. This is gonna this is gonna work this time. 100%. Here we go. Let's go. We are going to actually now we're gonna fetch that from format.payload. Actually, it's not, is it? Body.pricing URL. Nope. It's just nobody. There we go. Format .pricing URL should be good Or did I data dot pricing URL. Gotta help us. Format dot data. Alright. It is going to work this time. We are going to alright. So now we could see. Right? This is, we could we could actually set this up to be asynchronous as we wanted, but it is waiting for that to come back. And now bada bing bada boom. What do we have? This is the this is the data structure that we want. Right? Yeah. Exactly. Sick. Okay. Alright. So now we just need to basically wire that up to direct us. And in this case, it's going to be, let me make this full screen again. And we'll Additionally, just to make this workflow, more reliable and just continue applying changes if something was actually changed from the pricing page, we can use the change tracking. So this would be just an additional format that that would instruct our workflow, yeah, something changed, right, instead of always fetching. Yeah. But that's Yeah. There's plenty more things that we we can we can add just to make sure everything's on point. Yeah. Okay. Well, it let me let's make sure we can update the software first, and then we will then we'll tackle that, man. Alright. So we've got software. That's our collection. We are gonna I'm just gonna give this full access for now. And what we need to do, this should work here. We're gonna do something like trigger dot keys dot zero. So it trigger dot keys. This is just an array of the the primary key for, whatever we're triggering on. I'll hit enter there. And then the payload that we're going to send I'm gonna save this real quick. We're gonna look at what we received back. So it's just Navigate the tree. Dot JSON, basically. Yeah. So it should be payload Actually, dot dot data dot data dot data. Yeah. Dot data dot Are there the q dot data there? Oh, oh, I see JSON. Yeah. The it's like it's it's nested because the operation itself returns it in a data, and this is the fetch call that gets returned. So it's so it is data dot data dot JSON. JSON. Is what we want. Yep. Alright. So we should be able to do this where we have we're just gonna check the key here. I call this call fire crawl. I can always adjust these keys and that's, again, that's what it's gonna pin to. So it should be able to do something simple like this where we say firecrawl.data.data.json. And this is the pricing that we're going to update. I think it's surprising. What did we we'll open this up. What did I what did I call it here? Pricing data. Let me look at the actual data model real quick. Pricing data. That's that is indeed what it needs to be. Pricing underscore data. Alright. Now go back to the side by side, get that lovely shot of fire crawl. And now this is actually gonna work. Right? Yep. If the demo gods are on our side. Always in the demo. Here we go. Let's save this just just to make sure. Alright. Run flow. Now we should see this pricing change if everything works okay. Updated two fields. Something I'm assuming something failed. So it wasn't fire crawl. It was just me. BSS software. Keys undefined. Alright. Body dot keys. Okay. That's what it is. Operator error again. Body dot keys. Alright. One more time. Fetch pricing data. Oh, no. You're gonna use all my FireCrawl credits before we even use them, actually. Yeah. Sick. Okay. Now we could see it actually worked this time. So amazing. Right? Now we've got this wired up where it will update the pricing data for us on on this schedule, or well, actually, not schedule. It's totally manual. Right? That's still not what we want. Talk to me about the the change tracking bids. Yeah. Change tracking is it's a format that you'll specify just like we specify JSON as a format, we can add chain tracking. And inside of that chain tracking, like we fetched to JSON, there'll be the chain tracking object that will tell us, hey. Something changed in the website or, no. Nothing changed. So you don't really need to update anything. And that's just that's an add on. We don't we don't really need to implement this in here. But it it's nice that you mentioned about wasting your Firepower credits because the reason why we went for scraping, which, by the way, we can use batch scraping to do everything, all at once, do everything in parallel. Every scrape you do to to every single page wastes one single credit. Gotcha. And the more you abstract Yeah. Yeah. Yeah. And the the the more you abstract, the more, like, for example, if you use crawl, crawl is a combination of map with scrape, then, you'll like, the credits will, be higher for that just because you're fetching the different pages. If you then abstract away to using agents, since the agent executes everything for us, you don't really need to think about searching, scraping, mapping, or anything. It should, it will be a bit more expensive just because it's handling a bunch of things that you really don't need to decide. So Gotcha. Yeah. So it's like a sliding scale of what your what your your tolerance is of, like, hey. I know the exact workflow that I need. Exactly. You could you could be very surgical or, like, the agent Yes. Would just be, like, here's the URL. Here's some guidance. Here's the data that I want. Yes. Just work until you find that. Yes. Yeah. And and we can get creative with that. So, while you were implementing, I thought we could use agents to fetch for the pricing page or for whatever pages from that specific software, and then just place that in over back in Directus, place it in the database, and then always have Fireflies scrape for that given URL. So we would just be using agent to populate the URLs that will be scraped, not needing to use agent every single time. So we can get really creative with this, to ensure that we're only scraping at the times that we really need to. Yeah. Okay. Sick. Let's let's see how we do that. Right? I'm in the the fire crawl dashboard here. Is this something that's better done here, or is it Yeah. Like, in code or It's it's it's a nice place for us to test it. Okay. Yeah. Alright. In there, you can actually just type in it could be a very high level. So example, just get the name of the you don't even need to place the URL, to be honest. You can just place Okay. Place in the name of the software and say, fetch the pricing for this software. For let's see what we got. Let's do a different one. Let's say, core bridge here. First, the pricing for CoreBridge software. Yeah. We'll just say sign industry in case they've got there's multiple. Right? So what's the the difference between Spark Mini and Spark Pro? Spark Mini is so it's lightweight. It it's like the the comparison between using a Haiku model and a Claude or an op opus model. So one will be faster and the other will be just more efficient. Alright. Yeah. So here we are, Leo. We've seen, we've got this thing is it's fetching the pricing, and now it's asking for the details here. Is that, like, a failure for for me No. It's actually instructing this? Yeah. It's a default just to fetch for more information. Usually, we don't give enough context to LLMs. So it's just asking for that context, but let's we can skip that. There's a button to just skip and immediately so down there Okay. Yeah. I see it. We could just skip through it. It'll generate a specific schema that it should use. So it's already defining that schema. We always always try to output structured data. Yeah. Sick. And then now we and then Yes. Exactly. And then you can just run the agent, and it should fetch everything that we need. Since it it it runs asynchronously Uh-huh. You can ask for for it to alert us back whenever it's done. So let's see what it's come up with. Right? It's got core bridge starter, SMB, core bridge plus. It's it's came back with, like, eight plans. Oh, yeah. Let let's see if it's even, like, previous pricing for them. No. It looks like there are that many plans. Annual. Okay. So it's divided into monthly and annual. Okay. Six. So, you know, I have $1.99, $3.99, or $1.29. I'm sorry. 549 and 899. Yeah. This is so the data checks out, man. I think, you know, that's a a concern with everybody these days, especially on, like, the the agents. Do you what's the the secret sauce? Is there yeah. Do you care to share? Like, how do you guys get accurate results on this stuff? Oh, it's it's a mixture of system prompting using the best LLMs, understanding the flow of scraping. So Mhmm. Just having a fallback to when you need to interact with data, understanding, yeah, caching is is a part of it as well. So, yeah, it it's a bunch of I I I I feel like it's a bunch of little things that when summed up gets a pretty nice result like this. Yeah. Nice, man. Nice. Well, I know we're we're probably coming up on time. So, like, I won't go through the the trouble of integrating, like, the agent, but, you know, the fact that it would be, like, super simple here Yeah. Again, it's just a v two slash agent, and then we'll It would be the same process as before. Yeah. Yeah. Actually, it may not be that bad. Alright. So we go back to our flow. You know, I would probably change this as well, right, where instead of a manual trigger, we could go into, like, a cron, and I could trigger this, like, once a week or once a month or something like that. We'll we'll just keep it manual for now. But let's let's try this out and see. So we're gonna change the Yeah. Change it over to agent. And then it's just gonna be need to add a prompt and a a schema? Do we have the we got the prompt already? Right? And then the rest of this, we just remove? Nope. Yeah. I I think she can actually remove everything since the prompt was specific to the URL it was given. Yeah. We just need the the schema. Right? Alright. So we have prompts. Yep. Fetch the pricing for company name. Let's do the oh, I don't know how this is gonna work out. We'll see. And we can actually explore even more. I I mean, we don't have enough enough time to do so, but every single schema from your website, not only pricing, but could be handled from the agent. So we could have many different arrays and and objects to explore all the sections of that. Got you. Yeah. So it doesn't have to be, like, pricing. We could go in and basically, we're saying we could do the entirety of this and have Yes. Like, agent just continue. Yeah. Like like, attributes or any of that. Everything. Yep. Oh, man. That would be amazing. Exactly. Yeah. Sick. Okay. Well, hey. This is probably a good stopping point then. You know, on on fire crawl, man, like, you know, this is just as obviously, like, one simple workflow that we've touched on. Yeah. What's what's next for you guys? Or, like, what, is there anything you wanted to highlight out of the out of this? Let's see. I believe I believe FireCrawl. I think from this simple demo, it shows implementing something as simple as this. You get you get a a lot. Right? So from just one endpoint or one tool from Firegral, you can explore, the entire web. And there is so many things that we can build with this. Because before before AI, before all this wave of AI, scraping was something really difficult. Scraping was something that, you'd have to handle everything. You'd have to try to treat and filter out specific words, specific sessions. And now with FireCrawl, you have an LLM handling everything for you. It just allows us to build a lot more. And not only to build with FireCrawl because we we showed an example where we're using the FireCrawl API inside of our app. Right? Yeah. But even coding, and I I'm seeing directors implementing more and more AI tools. Eventually We are then. Yeah. Eventually, you could integrate Firewall into those agents as well and have Firewall have information, have much more context because whoever uses AI, enough understands that context is king. Right? And if you parrot it with the best web scraper out there, then you'll you'll have a lot of, pretty much best results. Yeah, man. But yeah. I mean Can we file or you guys have got the CLI now as well. I see. Right? Yes. That could be the next step for me is, like, oh, let's mess with this. Right? Because they like, my day to day right now is a a lot of quad code certainly. Yeah. And if the agent uses a CLI, it it can do a lot more. Right? Yeah. Yeah. Sick. Okay. Yeah. So yeah. Another follow-up session then then for sure where we're gonna take the direct us MCP, and then we will add the fire crawl CLI, and then we will just turn cloud code loose. Because everything that we did here today, like, you could do that via, the direct SMTP. So I could have Claude build a flow that just calls fire crawl agent and, you know, we could simplify all this work. But Yeah. It would be a matter of five minutes for it to build the entire page. Yeah, man. Sick. Sick. Cool. Well, Leo, man. And thanks for the the session. I I really appreciate you guys joining. Thanks. Thanks for for having me. Just one more just an additional tip. Follow our YouTube channel. Yeah. Our YouTube channel, they're we're we're posting everything there. We launch a lot. So to keep up with the scraping industry, feel free to subscribe there. Fire crawl YouTube. At fire crawl underscore dev. There you go. Yeah. Yeah. Sick. I see your face. There you are, man. Awesome, dude. Well, again, thanks for joining. This has been a fun session, and we could definitely follow-up again with a a run this back, man. Absolutely. Alright. Excited to do it again. Alright. See you. Thanks, everybody.","b984e4a4-f9d7-4e77-9200-906ac7f71e2d",[60,61],"739d3849-5a61-4b70-9022-97abe785813f","73deee8d-d24b-4cf9-a11f-a2adac9d0202",[],1781213212844]