gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > need help getting wifi to send more than ~20K of data

#171395 - Ant6n - Wed Nov 18, 2009 8:40 am

Hio,

In some project I need to send possibly up to like 500KB at the time, which I thought would not be a problem using today's ultra speeds of wifi's and dsl's. I use the http example from devkitpro with POST to get the data onto my server.
In other threads there is mention that 'send' only sends ~8K data at a time, so I do the resend trick, with a swiWaitForVBlank thrown in for good measure. The final code looks like this

Code:

char * url =
        "<host>";
char* request =
        "POST /<user>/cgi-bin/DSWifiSubmit.py HTTP/1.1\r\n"
        "<host>\r\n"
        "Content-Length: %d\n"
        "\n"
        "recording=%s";


int getHttp(char* url,char *request_text) {
  int success = 0;

  pause_terminal();//pause terminal -- sending might get disrupted by interrupts

  // Find the IP address of the server, with gethostbyname
  struct hostent * myhost = gethostbyname( url );
  iprintf("Found IP Address, ");
 
  // Create a TCP socket
  int my_socket;
  my_socket = socket( AF_INET, SOCK_STREAM, 0 );
  iprintf("Created Socket");

  // Tell the socket to connect to the IP address we found, on port 80 (HTTP)
  struct sockaddr_in sain;
  sain.sin_family = AF_INET;
  sain.sin_port = htons(80);
  sain.sin_addr.s_addr= *( (unsigned long *)(myhost->h_addr_list[0]) );
  connect( my_socket,(struct sockaddr *)&sain, sizeof(sain) );
  iprintf("Connected to server,sending data");

  // send our request - it only sends ~8k(?) bytes at a time
  int sent = 0, size = strlen(request_text);
  while(sent < size){
    printf("sent %d of %d bytes\n",sent,size);
    sent += send(my_socket, &(request_text[sent]), size - sent, 0);
    swiWaitForVBlank();
  }
  printf("sent %d of %d bytes\n",sent,size);
  iprintf("Sent data, printing result:\n");

  int recvd_len;
  char incoming_buffer[256];


  while( ( recvd_len = recv( my_socket, incoming_buffer, 255, 0 ) ) != 0 ) { // if recv returns 0, the socket has been closed.
    if(recvd_len>0) { // data was received!
      incoming_buffer[recvd_len] = 0; // null-terminate
      iprintf(incoming_buffer);
      resume_terminal();//reactivate terminal
      nextDownEvent();
      pause_terminal();//pause terminal -- sending might get disrupted by interrupts
    }
  }
  resume_terminal();//reactivate terminal

  iprintf("\nclosed connection\n");

  shutdown(my_socket,0); // good practice to shutdown the socket.
 
  iprintf("shutdown socket, ");

  closesocket(my_socket); // remove the socket.

  iprintf("closed socket\n");
   
  success = 1;
  return success;
}



The first time it sends 8K, then a bit less than 2K, apparently going down to about 1K, then it stops submitting. It doesn't always behave the same way. When I try with smaller data (~14K), it finishes succesfully and prints the result from the server. Ideas?

#171430 - Ant6n - Thu Nov 19, 2009 7:10 am

Some further research mentioned there might be something going on with some changes in the FIFO this year, but that all sounds like very low level to deal with.

So to deal with it, I am using a portable compression library, at http://oldhome.schmorp.de/marc/liblzf.html. It's nice because it has only two h and c files, and is portable. It's a bit crappy at compression, so with this it actually makes sense to compress twice; my data of mostly zeros compressed down to ~50% the first time, and down to 10% of that the second time.

#171433 - sverx - Thu Nov 19, 2009 10:34 am

why not zlib?

#171435 - Ant6n - Thu Nov 19, 2009 11:56 am

because it's hard to find libraries like that online (and in some previous thread a question for some lightweight portable library wasn't answered).

I don't like this zlib, it has way to many files, and within 5 minutes I didn't find functions for compressing and decompressing given pointers to memory. For people that already know it seems fine, or for people that might use it again later (which is not me).

That said, the library I mentioned works in the emulator for me, but it doesn't seem to work in hardware well, although there might be some other things going on here. I'll try another tiny, portable compressor library without dependency called minilzo http://www.oberhumer.com/opensource/lzo/.


Last edited by Ant6n on Thu Nov 19, 2009 8:22 pm; edited 2 times in total

#171439 - sverx - Thu Nov 19, 2009 12:40 pm

Ant6n wrote:
I don't like this zlib, it has way to many files, and within 5 minutes I didn't find functions for compressing and decompressing given pointers to memory.


There are compress() and uncompress()
http://zlib.ipinfo.de/manual.html#Utility functions
It's fast and has a good compression ratio. Has been used, for instance, in Manic Miner in the Lost Levels homebrew, to compress XM tunes stored in the .NDS and decompress them at run time.

#171442 - headspin - Thu Nov 19, 2009 2:15 pm

zlib is very easy to implement and work with as sverx said. If you need any help with it let me know.
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game

#171447 - Ant6n - Thu Nov 19, 2009 8:29 pm

huh, I guess I didn't find that in 5 minutes late yesterday night ;-).
This miniLZO works on the hardware btw, and it's pretty light weight to use.
I think I'll try Zlib as well; I have a feeling it should give me better compression ratios.

I presume it works if I just throw all the files of the top level directory of the library into my source code (except example.c, minizgip.c).

Thanks for the input :-)

#171526 - Izhido - Tue Nov 24, 2009 3:13 pm

Quote:

I don't like this ***, it has way to many files, and within 5 minutes I didn't find functions for *** and *** given ****.

Then why the hell are you here, acting as a software developer ???

</grumpy>

I think I need my morning coffee... urgently... :P