[{"data":1,"prerenderedAt":63},["ShallowReactive",2],{"quick-connect-netlify":3,"quick-connect-netlify-next":48},{"id":4,"slug":5,"vimeo_id":6,"description":7,"tile":8,"length":9,"resources":10,"people":11,"episode_number":15,"published":16,"title":17,"video_transcript_html":18,"video_transcript_text":19,"content":10,"status":20,"episode_people":21,"recommendations":32,"season":33,"seo":10},"71e081db-92f8-4978-b020-7d2460a46187","netlify","899795774","Increase visibility of build status by showing them in Directus.","ac4ea98a-a6ac-40c7-8c36-89c41992f27d",11,null,[12],{"name":13,"url":14},"Kevin Lewis","https:\u002F\u002Fdirectus.io\u002Fteam\u002Fkevin-lewis",6,"2024-02-02","Show Build Status with Netlify","\u003Cp>Speaker 0: Netlify is a platform that lets you build and deploy web applications. So it will take code, it will run any build scripts that are required, and then it will deploy the output of that build script. Now this episode of quick connect is all about increasing visibility into a specific deploy, so people know whether it has been successful or not directly from within directors. So people who author content or author updates to content can see when that content has been successfully deployed or perhaps when it hasn't because there's been a problem, and they know that there's some intervention needed. So let's get started.\u003C\u002Fp>\u003Cp>The first thing we're going to do is head over to flows, and we're going to create a new flow. I'll call this deploy updates, And we're gonna make this a webhook trigger with a post method. Once we create this flow, we are now given a URL. Whenever this URL receives a post request, the flow will begin. So now we're gonna head over to Netlify, and I've set up this little project here.\u003C\u002Fp>\u003Cp>Very minimal, but it does have a build step, which is important. And we're going to create a new deploy notification, which is an HTTP post request. And in here, we're gonna put the URL that we have just generated from our flow. Now when it comes to which event to listen for, we are going to pick deploy started. So this URL will receive an update whenever a deploy has started.\u003C\u002Fp>\u003Cp>We'll hit save. While we're here, we're also going to immediately add a second post request for when a deploy has succeeded, and we're gonna send that to exactly the same URL. We're just gonna do when they've started and succeeded today, but in reality, you'll want to set up a third one for when something has not succeeded, when there's been an error. This is actually all we need to do right here, right now. Let's head over to Directus.\u003C\u002Fp>\u003Cp>Actually, no. Let's not head over to Directus. Let's trigger a deploy and see what happens. So let's deploy our website. And what Netlify is now doing is it's going to grab the code.\u003C\u002Fp>\u003Cp>It's going to grab all of the dependencies, then it's going to execute the build script that I have provided, which should take almost no time whatsoever again because it's a very minimal site here. There we go. They'll all fill in, and now the site is live. So let's refresh our flow here. And I'm not sure what you might be expecting, but there are actually 2 invocations of this flow.\u003C\u002Fp>\u003Cp>The first one, if we look at the payload, has some interesting information. Firstly, it has a build ID. So this is for any given deploy, this build ID will stay the same. It has the name of the website, which is also useful because one Netlify account, could be hosting multiple websites, and you may want this flow for multiple sites. For example, at Directus, we have a website, and we have the dots, and they are both deployed by Netlify.\u003C\u002Fp>\u003Cp>And we may wanna reuse this flow to share the logic. We also have, what else is interesting here? The state. So at the beginning, a state is always building. That is the first step pair after initialization.\u003C\u002Fp>\u003Cp>Now if we look at the second invocation, it looks almost identical, except the state is now ready because the build succeeded. The state could also not be ready. It could indicate there's an error, should there be 1. But, otherwise, it's the same. The build ID remains the same.\u003C\u002Fp>\u003Cp>The name of the site stays the same. So first thing we're gonna do here is actually first thing we're gonna do is determine whether or not this is a brand new item or whether this is an update for an existing deploy. So what we'll do is we'll create a condition. I will put in this little default here, except I'm gonna change the values. So it's going to be Bobby.\u003C\u002Fp>\u003Cp>Inside of the body, there was a state, and we want to check whether that state is equal to, oops, building. Oh, building. Like so. If it is equal to building, that means this is a brand new entry. Otherwise, it means it's an update, and either there's been a success or a failure.\u003C\u002Fp>\u003Cp>Now in the case that this is successful, we want to store this new deploy in directors. So let's set up a collection for this. I'll call it deploys, and I'm going to make the primary key build underscore ID and make that a manually entered string. There are loads of values we can add here. I think what's useful for now is date created and date updated.\u003C\u002Fp>\u003Cp>Going into the data model, we want to add a state. You could, of course, chain covers as displayed. I'm just gonna make it a text input for now to keep it easy. The name of the website itself. And I think that's all we need, the build ID, the name of the site, and the state.\u003C\u002Fp>\u003Cp>Fantastic. So now let's go back to flows. And let's say when this is a brand new item, I e, building is true, we're going to create a new item in the deploys collection. We'll give it full access, and we're going to save 3 keys. Firstly, the build ID, which is required, is going to be equal to trigger dot body dot build underscore ID.\u003C\u002Fp>\u003Cp>Next was the state, and finally, the site. The order doesn't really matter here they'll all get saved in the correct fields. Great. I'm gonna hit save. I'm just gonna double check.\u003C\u002Fp>\u003Cp>Was it called site? No. It's called name. So, I will just go and edit that. I had a had a feeling it was wrong.\u003C\u002Fp>\u003Cp>So that's name, and the value is actually also name in the payload. So let's hit save, and let's retrigger a deploy. We'll just have to wait a moment for that deploy to start and then complete. And what we should see is a new item is created in the collection when it is first when the build first starts, And nothing should currently happen when we get the updated, when we get the update saying that it has been successful. So we see successful successful.\u003C\u002Fp>\u003Cp>Refresh here, and we should see 4 logs. Correct. The first one has been created in the collection. We see there. Quick connect demo building, and there's the build ID in Netlify.\u003C\u002Fp>\u003Cp>Now the latest one has a validation error because it was false, which means nothing then happened, which is fantastic. The last thing to really do here is on the reject route of the condition is we're gonna update data. We know it already exists because the first, the first payload we get for each deploy creates the item. Now we just get to update it, which is nice. So what we're gonna do here is we're going to update trigger dot Bobby dot build ID because we know that that is the unique identifier for the item in the collection.\u003C\u002Fp>\u003Cp>And all we're gonna do here is update state. Oops. So that's going to be, the trigger dot body dot state, which could be success already, or it could be something else. So let's hit save. Let's save again.\u003C\u002Fp>\u003Cp>Let's retrigger that deploy one more time. And you can imagine that this deploy isn't triggered manually. It's triggered because a user has updated something indirect or for some other reason, there is a a rebuild of the application that's taking place. So it's going ahead. It's building it.\u003C\u002Fp>\u003Cp>It's going through all those steps. If we look over here, we now see building. If I refresh again, still building. If I refresh again, still building. What's happened there?\u003C\u002Fp>\u003Cp>Let's take a look. Oh, you don't have permission to access this. This had incorrect permissions. This wants to be full access. Fantastic.\u003C\u002Fp>\u003Cp>So let's go, and we will now trigger that deploy one more time just to validate everything is set up correctly. So we should expect this to take 10 ish seconds, 10, 15 seconds like it has in the past. Basically, the moment it hits that build step is pretty instant just because of this specific applications. But builds can take, you know, 10 minutes sometimes, if necessary. So now if we look at this collection, we see that it's ready.\u003C\u002Fp>\u003Cp>So that's the 3rd deploy. We did one at the very start. We did one just now to test it, and that was a problem. And now that we see it's ready. So that is a little bit of, of information on how you can connect Netlify.\u003C\u002Fp>\u003Cp>We can also see date created, date updated. You know, we can sort by date created, date updated, whatever. And then, you can use this to find the deploy that's just happened and, share that with your team. Hope you found this in this episode of quick connect interesting. This general setup will work across hosting platforms.\u003C\u002Fp>\u003Cp>The Netlify is the one we use, so it's the one that I wanted to show off. Until next time, have a fantastic day, and I'll see you in the next episode.\u003C\u002Fp>","Netlify is a platform that lets you build and deploy web applications. So it will take code, it will run any build scripts that are required, and then it will deploy the output of that build script. Now this episode of quick connect is all about increasing visibility into a specific deploy, so people know whether it has been successful or not directly from within directors. So people who author content or author updates to content can see when that content has been successfully deployed or perhaps when it hasn't because there's been a problem, and they know that there's some intervention needed. So let's get started. The first thing we're going to do is head over to flows, and we're going to create a new flow. I'll call this deploy updates, And we're gonna make this a webhook trigger with a post method. Once we create this flow, we are now given a URL. Whenever this URL receives a post request, the flow will begin. So now we're gonna head over to Netlify, and I've set up this little project here. Very minimal, but it does have a build step, which is important. And we're going to create a new deploy notification, which is an HTTP post request. And in here, we're gonna put the URL that we have just generated from our flow. Now when it comes to which event to listen for, we are going to pick deploy started. So this URL will receive an update whenever a deploy has started. We'll hit save. While we're here, we're also going to immediately add a second post request for when a deploy has succeeded, and we're gonna send that to exactly the same URL. We're just gonna do when they've started and succeeded today, but in reality, you'll want to set up a third one for when something has not succeeded, when there's been an error. This is actually all we need to do right here, right now. Let's head over to Directus. Actually, no. Let's not head over to Directus. Let's trigger a deploy and see what happens. So let's deploy our website. And what Netlify is now doing is it's going to grab the code. It's going to grab all of the dependencies, then it's going to execute the build script that I have provided, which should take almost no time whatsoever again because it's a very minimal site here. There we go. They'll all fill in, and now the site is live. So let's refresh our flow here. And I'm not sure what you might be expecting, but there are actually 2 invocations of this flow. The first one, if we look at the payload, has some interesting information. Firstly, it has a build ID. So this is for any given deploy, this build ID will stay the same. It has the name of the website, which is also useful because one Netlify account, could be hosting multiple websites, and you may want this flow for multiple sites. For example, at Directus, we have a website, and we have the dots, and they are both deployed by Netlify. And we may wanna reuse this flow to share the logic. We also have, what else is interesting here? The state. So at the beginning, a state is always building. That is the first step pair after initialization. Now if we look at the second invocation, it looks almost identical, except the state is now ready because the build succeeded. The state could also not be ready. It could indicate there's an error, should there be 1. But, otherwise, it's the same. The build ID remains the same. The name of the site stays the same. So first thing we're gonna do here is actually first thing we're gonna do is determine whether or not this is a brand new item or whether this is an update for an existing deploy. So what we'll do is we'll create a condition. I will put in this little default here, except I'm gonna change the values. So it's going to be Bobby. Inside of the body, there was a state, and we want to check whether that state is equal to, oops, building. Oh, building. Like so. If it is equal to building, that means this is a brand new entry. Otherwise, it means it's an update, and either there's been a success or a failure. Now in the case that this is successful, we want to store this new deploy in directors. So let's set up a collection for this. I'll call it deploys, and I'm going to make the primary key build underscore ID and make that a manually entered string. There are loads of values we can add here. I think what's useful for now is date created and date updated. Going into the data model, we want to add a state. You could, of course, chain covers as displayed. I'm just gonna make it a text input for now to keep it easy. The name of the website itself. And I think that's all we need, the build ID, the name of the site, and the state. Fantastic. So now let's go back to flows. And let's say when this is a brand new item, I e, building is true, we're going to create a new item in the deploys collection. We'll give it full access, and we're going to save 3 keys. Firstly, the build ID, which is required, is going to be equal to trigger dot body dot build underscore ID. Next was the state, and finally, the site. The order doesn't really matter here they'll all get saved in the correct fields. Great. I'm gonna hit save. I'm just gonna double check. Was it called site? No. It's called name. So, I will just go and edit that. I had a had a feeling it was wrong. So that's name, and the value is actually also name in the payload. So let's hit save, and let's retrigger a deploy. We'll just have to wait a moment for that deploy to start and then complete. And what we should see is a new item is created in the collection when it is first when the build first starts, And nothing should currently happen when we get the updated, when we get the update saying that it has been successful. So we see successful successful. Refresh here, and we should see 4 logs. Correct. The first one has been created in the collection. We see there. Quick connect demo building, and there's the build ID in Netlify. Now the latest one has a validation error because it was false, which means nothing then happened, which is fantastic. The last thing to really do here is on the reject route of the condition is we're gonna update data. We know it already exists because the first, the first payload we get for each deploy creates the item. Now we just get to update it, which is nice. So what we're gonna do here is we're going to update trigger dot Bobby dot build ID because we know that that is the unique identifier for the item in the collection. And all we're gonna do here is update state. Oops. So that's going to be, the trigger dot body dot state, which could be success already, or it could be something else. So let's hit save. Let's save again. Let's retrigger that deploy one more time. And you can imagine that this deploy isn't triggered manually. It's triggered because a user has updated something indirect or for some other reason, there is a a rebuild of the application that's taking place. So it's going ahead. It's building it. It's going through all those steps. If we look over here, we now see building. If I refresh again, still building. If I refresh again, still building. What's happened there? Let's take a look. Oh, you don't have permission to access this. This had incorrect permissions. This wants to be full access. Fantastic. So let's go, and we will now trigger that deploy one more time just to validate everything is set up correctly. So we should expect this to take 10 ish seconds, 10, 15 seconds like it has in the past. Basically, the moment it hits that build step is pretty instant just because of this specific applications. But builds can take, you know, 10 minutes sometimes, if necessary. So now if we look at this collection, we see that it's ready. So that's the 3rd deploy. We did one at the very start. We did one just now to test it, and that was a problem. And now that we see it's ready. So that is a little bit of, of information on how you can connect Netlify. We can also see date created, date updated. You know, we can sort by date created, date updated, whatever. And then, you can use this to find the deploy that's just happened and, share that with your team. Hope you found this in this episode of quick connect interesting. This general setup will work across hosting platforms. The Netlify is the one we use, so it's the one that I wanted to show off. Until next time, have a fantastic day, and I'll see you in the next episode.","published",[22],{"people_id":23},{"id":24,"first_name":25,"last_name":26,"avatar":27,"bio":28,"links":29},"82b3f7e5-637b-4890-93b2-378b497d5dc6","Kevin","Lewis","a662f91b-1ee9-4277-8c9d-3ac1878e44ad","Director of Developer Experience at Directus",[30],{"url":14,"service":31},"website",[],{"id":34,"number":35,"year":36,"episodes":37,"show":45},"3b8b7d34-a0fb-4ea6-85ff-2b5bfbb8e0b6",1,"2023",[38,39,40,41,42,4,43,44],"502dcf7e-c23e-4dfd-b147-65f5abaea5c7","a230c9ef-8db4-4c00-a0cb-9524f7934eb0","5f41dc16-29b7-485f-a6e1-081c3f1acc4f","81417d25-26d2-4f05-be37-7ced51a0594e","8f933ee9-4e4f-4e35-8c1f-e99ad0684bfa","8e47020d-bd5a-43a7-bca9-54af4f5d465d","bfb8bc25-ef1b-4544-b50d-402008c638a1",{"title":46,"tile":47},"Quick Connect","1171b046-491e-4cfb-a68c-527b89c2c348",{"id":49,"slug":50,"season":51,"vimeo_id":52,"description":53,"tile":54,"length":55,"resources":10,"people":10,"episode_number":35,"published":56,"title":53,"video_transcript_html":57,"video_transcript_text":58,"content":10,"seo":59,"status":20,"episode_people":60,"recommendations":62},"979db4da-a870-4120-94ee-bd80789f411c","firecrawl","cf7a056d-fa10-4bc5-8cc3-c2b9ef59b684","1026203173","Integrating Firecrawl with Directus","d0a87153-8475-433f-aca0-dea9802caf03",10,"2024-10-16","\u003Cp>Speaker 0: Hello there. I'm really excited about this tutorial. So on Directus TV, we already have a show called Quick Connect, which shows you how to integrate third party services with Directus using Directus Automate and Flows. And in the spirit of that show, today, I'm gonna show you how to integrate FireCrawl with Directus. Now here they say that they turn websites into LLM ready data.\u003C\u002Fp>\u003Cp>And what that means in practice is you can feed it a URL, provide some options if you want, and it will go and take a look at that web page and return some structured data for you like so. This is their scrape endpoint, which will take a single web page and scrape some data from it. They also have a couple of other endpoints, crawl and map, but today, we're gonna use scrape. Now I've already logged into FireCrawl Cloud and generated an API key, which I'll copy for later. You can also self host FireCrawl, but for ease, I'm just gonna use their cloud product here.\u003C\u002Fp>\u003Cp>Now I have this directors project over here with a new empty collection called companies. In this collection, there are a few fields. URL, a name, a description, mission, and a boolean, a true force value, is it open source. And our goal will be to provide the URL and then have FireCrawl automatically populate the rest with flows. So let's go ahead and create a new flow.\u003C\u002Fp>\u003Cp>So this is our automation builder if you've not seen it before. I'll call this one get company data, I guess. And we are going to use a manual trigger, which will add a button to the side of collection and item pages. So we're going to say we'll run this on the company's collection. What else matters here?\u003C\u002Fp>\u003Cp>We're going to, not require selection, so the button always works. And we're going to require confirmation, which will pop up a modal. And in that modal, we will just add a URL. We'll make it a string input, and we'll make it full widths. And I think that's all we need to do here.\u003C\u002Fp>\u003Cp>So just to see what happens here, if I go back to the, to the company's collection, we now see this button here, this manual flow, trigger. I click that. It pulls up the box, and we'll put in a URL and hit run flow. So now if we go back to our flow, we should immediately see that there is one log. And in here, there is a body, and the URL is the value that we typed.\u003C\u002Fp>\u003Cp>Fantastic. Now we need to actually do something with it. So, let's go ahead and add a new operation here. And, honestly, fire crawl is pretty sick. You can just make one web request.\u003C\u002Fp>\u003Cp>Let's take a look at their docs. We're gonna use the LLM extract, endpoint here, and let's just take a look at the kind of construction of this API call. It's a post request to this URL. We're gonna pass in our, our API key here as an authorization header, and then they give us this kind of JSON payload here. Here, it's telling it to go ahead and extract specifically these four fields, the company mission, does it support SSO, is it open source, and is it in Y Combinator?\u003C\u002Fp>\u003Cp>And it's saying you must go get all four of these. So let's actually just turn this straight into a flow request, request URL, operation. So we're gonna do a post request to this URL, post request to this URL. I'm gonna go and copy my API key again here, and at the end of this I'll, I'll destroy the key. Authorization authorization, bearer API key, save, And then there's the request body.\u003C\u002Fp>\u003Cp>And, honestly, it contains a little more than we need, but this contains everything we need. So we'll just pop that in there. The only thing we wanna do, of course, is pass in the URL that we put in the box. So we'll replace this with trigger.body. URL.\u003C\u002Fp>\u003Cp>Fantastic. Let's save that and see what happens if we go over to content, press the button, and type in directors.io. We see that's running. That's running. That's a good sign.\u003C\u002Fp>\u003Cp>It means it's going off and making the request, waiting for the request. And then we see there is a second log. And we get some data back. There was a 200, so it was successful. Inside of data, there is a property called data, and then there is this value called extract.\u003C\u002Fp>\u003Cp>Extract contains all of those custom keys we asked for, company mission, supports SSO, is open source, and is in YC. And then always when you scrape, you get this metadata object, title, description, language, Open Graph data, source URL, and so on. So, really, all we wanna do here now is we wanna take this data and create a new company from it. So let's add a new, let's create a new operation on the resolve path of that web request. Let's call it create data.\u003C\u002Fp>\u003Cp>We're gonna create something in the company's collection. We'll give it full access, and then we just need to provide a payload. So let's go ahead and do that. We have an object here. We have a name.\u003C\u002Fp>\u003Cp>Oh, we have a name, and we're gonna pass in the value of the last operation dot data.data.metadata dot title. Then I, for one, am I'm just gonna copy this and edit it each time. So we have name, URL. So that's last .data.data.metadata. Source URL.\u003C\u002Fp>\u003Cp>We could, of course, just take it from the trigger body URL, but this is properly formatted. You'll notice I typed in directus dot I o, but when it came back in the payload, it came back with the with the, with the protocol HTTPS and so on. So we have name. We have URL. We have a description.\u003C\u002Fp>\u003Cp>Now this one is also from the metadata description. We have the mission. Now this was a custom piece of data we asked to be extracted. Company mission is what we called it. And finally, we have open, open underscore source.\u003C\u002Fp>\u003Cp>Last data, data extract and then is underscore open underscore source and then remove that trailing comma. So I believe that's the name of all of the fields. We'll figure it out in a moment when when it inevitably doesn't work. We'll hit the button again, directors.io, and hit run flow. Once again, that's going off to fire crawl using their endpoint and there we see there we see it straight here.\u003C\u002Fp>\u003Cp>URL name, description, mission, and the boolean is open source. Let's, let's try that once more. Let's go in here and say firefirecrawl.devrun flow. So let's see. And in theory, we should just give that a moment, and there it is.\u003C\u002Fp>\u003Cp>So now you can go ahead and grab more data. Now, of course, if we take a look at this endpoint here, you can provide custom properties, and it will try its best to get data out from that. They have a couple of other interesting things which I'll draw your attention to even if I don't think it works in this context. They have extracting without a schema. So this extract here was us creating a a schema.\u003C\u002Fp>\u003Cp>Right? You can give it just a text. You can give it a prompt. Extract the company mission from the page. But the thing I don't like about that is you're not explicitly saying what the name of the key is, so you don't necessarily know what it's gonna be at the end.\u003C\u002Fp>\u003Cp>I like creating a schema personally. They do something else that's kinda interesting. If I take a look at where is I think it's in their API API reference here inside of scrape. They have this interesting thing called actions. So you can get it to wait, to take a screenshot, to click, write text, press a key, and scroll.\u003C\u002Fp>\u003Cp>And the combination of clicking and writing text means you can get it to interact with your web page. You see it here, actions, wait two milliseconds. You could get it to, like, sign into things perhaps, perform search terms. I think it's super interesting. And then take screenshots, of course, and upload those to directors if you fancy.\u003C\u002Fp>\u003Cp>So there's a lot of flexibility in this. Having seen kinda how easy this API is, I think I'll go ahead and turn this into, an extension some point in the next few weeks, which we can release as part of Directus AI. But, yeah, that's how to integrate FireCrawl with Directus using Directus Automate. Hope you found this interesting, and by all means, if you have questions, just reach out.\u003C\u002Fp>","Hello there. I'm really excited about this tutorial. So on Directus TV, we already have a show called Quick Connect, which shows you how to integrate third party services with Directus using Directus Automate and Flows. And in the spirit of that show, today, I'm gonna show you how to integrate FireCrawl with Directus. Now here they say that they turn websites into LLM ready data. And what that means in practice is you can feed it a URL, provide some options if you want, and it will go and take a look at that web page and return some structured data for you like so. This is their scrape endpoint, which will take a single web page and scrape some data from it. They also have a couple of other endpoints, crawl and map, but today, we're gonna use scrape. Now I've already logged into FireCrawl Cloud and generated an API key, which I'll copy for later. You can also self host FireCrawl, but for ease, I'm just gonna use their cloud product here. Now I have this directors project over here with a new empty collection called companies. In this collection, there are a few fields. URL, a name, a description, mission, and a boolean, a true force value, is it open source. And our goal will be to provide the URL and then have FireCrawl automatically populate the rest with flows. So let's go ahead and create a new flow. So this is our automation builder if you've not seen it before. I'll call this one get company data, I guess. And we are going to use a manual trigger, which will add a button to the side of collection and item pages. So we're going to say we'll run this on the company's collection. What else matters here? We're going to, not require selection, so the button always works. And we're going to require confirmation, which will pop up a modal. And in that modal, we will just add a URL. We'll make it a string input, and we'll make it full widths. And I think that's all we need to do here. So just to see what happens here, if I go back to the, to the company's collection, we now see this button here, this manual flow, trigger. I click that. It pulls up the box, and we'll put in a URL and hit run flow. So now if we go back to our flow, we should immediately see that there is one log. And in here, there is a body, and the URL is the value that we typed. Fantastic. Now we need to actually do something with it. So, let's go ahead and add a new operation here. And, honestly, fire crawl is pretty sick. You can just make one web request. Let's take a look at their docs. We're gonna use the LLM extract, endpoint here, and let's just take a look at the kind of construction of this API call. It's a post request to this URL. We're gonna pass in our, our API key here as an authorization header, and then they give us this kind of JSON payload here. Here, it's telling it to go ahead and extract specifically these four fields, the company mission, does it support SSO, is it open source, and is it in Y Combinator? And it's saying you must go get all four of these. So let's actually just turn this straight into a flow request, request URL, operation. So we're gonna do a post request to this URL, post request to this URL. I'm gonna go and copy my API key again here, and at the end of this I'll, I'll destroy the key. Authorization authorization, bearer API key, save, And then there's the request body. And, honestly, it contains a little more than we need, but this contains everything we need. So we'll just pop that in there. The only thing we wanna do, of course, is pass in the URL that we put in the box. So we'll replace this with trigger.body. URL. Fantastic. Let's save that and see what happens if we go over to content, press the button, and type in directors.io. We see that's running. That's running. That's a good sign. It means it's going off and making the request, waiting for the request. And then we see there is a second log. And we get some data back. There was a 200, so it was successful. Inside of data, there is a property called data, and then there is this value called extract. Extract contains all of those custom keys we asked for, company mission, supports SSO, is open source, and is in YC. And then always when you scrape, you get this metadata object, title, description, language, Open Graph data, source URL, and so on. So, really, all we wanna do here now is we wanna take this data and create a new company from it. So let's add a new, let's create a new operation on the resolve path of that web request. Let's call it create data. We're gonna create something in the company's collection. We'll give it full access, and then we just need to provide a payload. So let's go ahead and do that. We have an object here. We have a name. Oh, we have a name, and we're gonna pass in the value of the last operation dot data.data.metadata dot title. Then I, for one, am I'm just gonna copy this and edit it each time. So we have name, URL. So that's last .data.data.metadata. Source URL. We could, of course, just take it from the trigger body URL, but this is properly formatted. You'll notice I typed in directus dot I o, but when it came back in the payload, it came back with the with the, with the protocol HTTPS and so on. So we have name. We have URL. We have a description. Now this one is also from the metadata description. We have the mission. Now this was a custom piece of data we asked to be extracted. Company mission is what we called it. And finally, we have open, open underscore source. Last data, data extract and then is underscore open underscore source and then remove that trailing comma. So I believe that's the name of all of the fields. We'll figure it out in a moment when when it inevitably doesn't work. We'll hit the button again, directors.io, and hit run flow. Once again, that's going off to fire crawl using their endpoint and there we see there we see it straight here. URL name, description, mission, and the boolean is open source. Let's, let's try that once more. Let's go in here and say firefirecrawl.devrun flow. So let's see. And in theory, we should just give that a moment, and there it is. So now you can go ahead and grab more data. Now, of course, if we take a look at this endpoint here, you can provide custom properties, and it will try its best to get data out from that. They have a couple of other interesting things which I'll draw your attention to even if I don't think it works in this context. They have extracting without a schema. So this extract here was us creating a a schema. Right? You can give it just a text. You can give it a prompt. Extract the company mission from the page. But the thing I don't like about that is you're not explicitly saying what the name of the key is, so you don't necessarily know what it's gonna be at the end. I like creating a schema personally. They do something else that's kinda interesting. If I take a look at where is I think it's in their API API reference here inside of scrape. They have this interesting thing called actions. So you can get it to wait, to take a screenshot, to click, write text, press a key, and scroll. And the combination of clicking and writing text means you can get it to interact with your web page. You see it here, actions, wait two milliseconds. You could get it to, like, sign into things perhaps, perform search terms. I think it's super interesting. And then take screenshots, of course, and upload those to directors if you fancy. So there's a lot of flexibility in this. Having seen kinda how easy this API is, I think I'll go ahead and turn this into, an extension some point in the next few weeks, which we can release as part of Directus AI. But, yeah, that's how to integrate FireCrawl with Directus using Directus Automate. Hope you found this interesting, and by all means, if you have questions, just reach out.","9c575c49-4fb1-4eed-9731-e1e1acb55da3",[61],"4830ae03-22cd-4399-82f2-5767bf5cf0cc",[],1781877756396]