c - Why does popen2() hang between write and read calls? -

c - Why does popen2() hang between write and read calls? -

i trying integrate utilize of samtools c program. application reads info in binary format called bam, e.g. stdin:

$ cat foo.bam | samtools view -h - ...

(i realize useless utilize of cat, i'm showing how bam file's bytes can piped samtools on command line. these bytes come other upstream processes.)

within c program, write chunks of unsigned char bytes samtools binary, while simultaneously capturing standard output samtools after processes these bytes.

because cannot utilize popen() simultaneously write , read process, looked using publicly-available implementations of popen2(), appears written back upwards this.

i wrote next test code, attempts write() 4 kb chunks bytes of bam file located in same directory samtools process. read()s bytes output of samtools line buffer, printed standard error:

#include <sys/types.h> #include <fcntl.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #define read 0 #define write 1 pid_t popen2(const char *command, int *infp, int *outfp) { int p_stdin[2], p_stdout[2]; pid_t pid; if (pipe(p_stdin) != 0 || pipe(p_stdout) != 0) homecoming -1; pid = fork(); if (pid < 0) homecoming pid; else if (pid == 0) { close(p_stdin[write]); dup2(p_stdin[read], read); close(p_stdout[read]); dup2(p_stdout[write], write); execl("/bin/sh", "sh", "-c", command, null); perror("execl"); exit(1); } if (infp == null) close(p_stdin[write]); else *infp = p_stdin[write]; if (outfp == null) close(p_stdout[read]); else *outfp = p_stdout[read]; homecoming pid; } int main(int argc, char **argv) { int infp, outfp; /* set samtools read stdin */ if (popen2("samtools view -h -", &infp, &outfp) <= 0) { printf("unable exec samtools\n"); exit(1); } const char *fn = "foo.bam"; file *fp = null; fp = fopen(fn, "r"); if (!fp) exit(-1); unsigned char buf[4096]; char line_buf[65536] = {0}; while(1) { size_t n_bytes = fread(buf, sizeof(buf[0]), sizeof(buf), fp); fprintf(stderr, "read\t-> %08zu bytes fp\n", n_bytes); write(infp, buf, n_bytes); fprintf(stderr, "wrote\t-> %08zu bytes samtools process\n", n_bytes); read(outfp, line_buf, sizeof(line_buf)); fprintf(stderr, "output\t-> \n%s\n", line_buf); memset(line_buf, '\0', sizeof(line_buf)); if (feof(fp) || ferror(fp)) { break; } } homecoming 0; }

(for local re-create of foo.bam, here link binary file using testing. bam file testing purposes.)

to compile:

$ cc -wall test_bam.c -o test_bam

the problem procedure hangs after write() call:

$ ./test_bam read -> 00004096 bytes fp wrote -> 00004096 bytes samtools process [bam_header_read] eof marker absent. input truncated.

if close() infp variable after write() call, loop goes through 1 more iteration before hanging:

... write(infp, buf, n_bytes); close(infp); /* <---------- added after write() phone call */ fprintf(stderr, "wrote\t-> %08zu bytes samtools process\n", n_bytes); ...

with close() statement:

$ ./test_bam read -> 00004096 bytes fp wrote -> 00004096 bytes samtools process [bam_header_read] eof marker absent. input truncated. [main_samview] truncated file. output -> @hd vn:1.0 so:coordinate @sq sn:seq1 ln:5000 @sq sn:seq2 ln:5000 @co illustration of sam/bam file format. read -> 00004096 bytes fp wrote -> 00004096 bytes samtools process

with change, output i'd otherwise expect if ran samtools on command line, mentioned, procedure hangs 1 time again.

how 1 utilize popen2() write , read info in chunks internal buffers? if isn't possible, there alternatives popen2() work improve task?

as alternative pipe, why not communicate samtools through socket? checking samtools source, file knetfile.c indicates samtools has socket communications available:

#include "knetfile.h" /* in winsock.h, type of socket socket, is: "typedef * u_int socket". invalid socket is: "(socket)(~0)", or signed * integer -1. in knetfile.c, utilize "int" socket type * throughout. should improved avoid confusion. * * in linux/mac, recv() , read() same thing. can see * in header file netread() alias of read(). in * windows, however, different , using recv() mandatory. */

that may provide improve alternative using pipe2.

c popen samtools