Using the libraries and APIs is better unless you need the flexibility. Don't call an external program unless you need to. On top of the pain of fork and execing, it's also much more expensive and far less efficient.
Also, curl is just a thin wrapper around libcurl anyway.
It's hard to know what would be the proper way exactly without knowing your exact use case. The best option, though, is to either use libcurl and readdir/readdir_r for those two uses, or luacurl and luafilesystem, depending on which part of your program and how integrated you want that functionality to be.
You usually don't want to fork and exec unless you want to be able to invoke external functionality dynamically, such as having the user specify a command to use (like with a shell, or with any system that needs user plugins that should be language-agnostic, can work with program output, and don't need real IPC).
Don't call an external command unless you really need to and it fits your workflow. Doing an external call to something like curl or find is redundant, pointless, and prone to way more bugs. Not to mention that the overhead of managing the forked process as a child and making sure it exits properly and everything that goes with it is way more work than just using libcurl and readdir properly.
Here's the basic way with a fork:
int pipefd[2];
pipe(pipefd);
pid_t parent = getpid();
pid_t pid = fork();
if (pid == -1)
{
// Some error forking
return 1;
} else if (pid > 0)
{
// I am the parent
// Close writing end
close(pipefd[1]);
ssize_t newstart = 0;
char buffer[BUFSIZE];
while (true)
{
// Do the BUFSIZE - newstart - 2 for necessary injected characters, if needed
ssize_t size = read(pipefd[0], buffer + newstart, BUFSIZE - newstart - 2);
// This is done because the buffer was already offset. We want size