gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

Coding > arm-eabi-as.exe SLOW

#164522 - Chano Marrano - Wed Nov 05, 2008 2:43 am

Using devKitArm r23b, when I include about 70 .mod songs in my project (I'm using AAS for sound), as.exe takes about 5 minutes to finish its execution. The project only contains a pair of .c files and the .mod files. The makefile i'm using is a mix between TONC makefile and the makefile included in devKitPro's AAS example:
Code:
# ---------------------------------------------------------------------
# SETUP
# ---------------------------------------------------------------------

# --- No implicit rules ---
.SUFFIXES:

# --- Tonc paths ---
# If not defined as environment var, assumed to be 2 dirs up
export TONCCODE   ?= C:/TONC

include $(TONCCODE)/tonc_rules

# --- Main path ---
export PATH   :=   $(DEVKITARM)/bin:$(PATH)


# ---------------------------------------------------------------------
# PROJECT DETAILS
# ---------------------------------------------------------------------

# PROJ      : Base project name
# TITLE      : Title for ROM header (12 characters)
# LIBS      : Libraries to use, formatted as list for linker flags
# BUILD      : Directory for build process temporaries. Should NOT be empty!
# SRCDIRS   : List of source file directories
# DATADIRS   : List of data file directories
# INCDIRS   : List of header file directories
# LIBDIRS   : List of library directories
# General note: use . for the current dir, don't leave them empty.

export PROJ   ?= $(notdir $(CURDIR))
TITLE      := $(PROJ)
GFXLIBS      := libgfx.a
LIBAAS      := $(TONCCODE)/libAAS

LIBS      := -ltonc -lgfx -lAAS

BUILD      := build
SRCDIRS      := source
DATADIRS   := data
INCDIRS      := source
LIBDIRS      := $(TONCCODE)/tonclib $(LIBAAS)
AASDATA      := audio

EMU := $(TONCCODE)/emu/emu.exe


# --- switches ---

bMB      := 0   # Multiboot build
bTEMPS   := 0   # Save gcc temporaries (.i and .s files)
bDEBUG2   := 0   # Generate debug info (bDEBUG2? Not a full DEBUG flag. Yet)


# ---------------------------------------------------------------------
# BUILD FLAGS
# ---------------------------------------------------------------------

# This is probably where you can stop editing

# --- Architecture ---

ARCH    := -mthumb-interwork -mthumb
RARCH   := -mthumb-interwork -mthumb
IARCH   := -mthumb-interwork -marm -mlong-calls

# --- Main flags ---

CFLAGS   := -mcpu=arm7tdmi -mtune=arm7tdmi $(ARCH) -O2
CFLAGS   += -Wall
CFLAGS   += $(INCLUDE)
CFLAGS   += -ffast-math -fno-strict-aliasing

CXXFLAGS   := $(CFLAGS) -fno-rtti -fno-exceptions

ASFLAGS   := $(ARCH)
LDFLAGS := $(ARCH) -Wl,-Map,$(PROJ).map

# --- switched additions ----------------------------------------------

# --- Multiboot ? ---
ifeq ($(strip $(bMB)), 1)
   TARGET   := $(PROJ).mb
else
   TARGET   := $(PROJ)
endif
   
# --- Save temporary files ? ---
ifeq ($(strip $(bTEMPS)), 1)
   CFLAGS   += -save-temps
endif

# --- Debug info ? ---

ifeq ($(strip $(bDEBUG2)), 1)
   CFLAGS   += -g
   LDFLAGS   += -g
endif


# ---------------------------------------------------------------------
# BUILD PROCEDURE
# ---------------------------------------------------------------------

ifneq ($(BUILD),$(notdir $(CURDIR)))

# Still in main dir:
# * Define/export some extra variables
# * Invoke this file again from the build dir
# PONDER: what happens if BUILD == "" ?

export OUTPUT   :=   $(CURDIR)/$(TARGET)
export VPATH   :=                           \
   $(foreach dir, $(SRCDIRS) , $(CURDIR)/$(dir))   \
   $(foreach dir, $(DATADIRS), $(CURDIR)/$(dir))   \
   $(CURDIR)/$(AASDATA)

export DEPSDIR   :=   $(CURDIR)/$(BUILD)

# --- List source and data files ---

CFILES      :=   $(foreach dir, $(SRCDIRS) , $(notdir $(wildcard $(dir)/*.c)))
CPPFILES   :=   $(foreach dir, $(SRCDIRS) , $(notdir $(wildcard $(dir)/*.cpp)))
SFILES      :=   $(foreach dir, $(SRCDIRS) , $(notdir $(wildcard $(dir)/*.s)))
BINFILES   :=   $(foreach dir, $(DATADIRS), $(notdir $(wildcard $(dir)/*.*)))
AAS_FILES   :=   $(notdir $(wildcard $(AASDATA)/*.*))

export AAS_DEPS   :=   $(foreach file,$(AAS_FILES),$(CURDIR)/$(AASDATA)/$(file))
export AAS_DIR   :=   $(CURDIR)/$(AASDATA)

# --- Set linker depending on C++ file existence ---
ifeq ($(strip $(CPPFILES)),)
   export LD   := $(CC)
else
   export LD   := $(CXX)
endif

# --- Define object file list ---
export OFILES   :=                           \
   AAS_Data.o                              \
   $(addsuffix .o, $(BINFILES))               \
   $(CFILES:.c=.o) $(CPPFILES:.cpp=.o)            \
   $(SFILES:.s=.o)

# --- Create include and library search paths ---
export INCLUDE   :=                           \
   $(foreach dir,$(INCDIRS),-I$(CURDIR)/$(dir))   \
   $(foreach dir,$(LIBDIRS),-I$(dir)/include)      \
   -I$(CURDIR)/$(BUILD)

export LIBPATHS   :=   -L$(CURDIR) $(foreach dir,$(LIBDIRS),-L$(dir)/lib)


# --- More targets ----------------------------------------------------

.PHONY: $(BUILD) clean

# --- Create $(BUILD) if necessary, and run this makefile from there ---

$(BUILD):
   @[ -d $@ ] || mkdir -p $@
   @make --no-print-directory -f $(CURDIR)/gfxmake
   @make --no-print-directory -C $(BUILD) -f $(CURDIR)/Makefile
   @$(EMU) $(TARGET).gba

all   : $(BUILD)

clean:
   @echo clean ...
   @rm -fr $(BUILD) $(TARGET).elf $(TARGET).gba


else      # If we're here, we should be in the BUILD dir

DEPENDS   :=   $(OFILES:.o=.d)

# --- Main targets ----

$(OUTPUT).gba   :   $(OUTPUT).elf

$(OUTPUT).elf   :   $(OFILES) libgfx.a

AAS_Data.s: $(AAS_DEPS)
   $(LIBAAS)/bin/conv2aas.exe $(AAS_DIR)

-include $(DEPENDS)


endif      # End BUILD switch

# EOF


So... any solution?

EDIT:
as.exe almost hangs at this line of execution:
Code:
AAS_Data.s
arm-eabi-gcc -MMD -MP -MF /c/ProyectoTSC/build/AAS_Data.d -x assembler-with-cpp -mthumb-interwork -mthumb -c AAS_Data.s -o AAS_Data.o

_________________
Sh*tty RPG

#164544 - Miked0801 - Thu Nov 06, 2008 6:54 pm

Does turing off the dependency check help out? Also, how big is said assembler file? Finally, if you just run the file without make overhead, how fast does it go?

#164545 - Chano Marrano - Thu Nov 06, 2008 9:38 pm

Quote:
Does turing off the dependency check help out?
I suppose you mean to do this:
Code:
AAS_Data.s:
   $(LIBAAS)/bin/conv2aas.exe $(AAS_DIR)
And nope, it doesn't help.


Quote:
how big is said assembler file?
About 30 MB.


Quote:
if you just run the file without make overhead, how fast does it go?
What file? I didn't understand that.

I don't understand why an assembler can be so slow. AAS converter (a more complex program, I suppose) takes less than a minute to convert all songs.


EDIT:
I tried to run this line on cmd.exe and the results were more or less the same:
Code:
arm-eabi-gcc -MMD -MP -MF /c/ProyectoTSC/build/AAS_Data.d -x assembler-with-cpp -mthumb-interwork -mthumb -c AAS_Data.s -o AAS_Data.o

Also I turned off the antivirus and assigned 1GB to the virtual machine (I'm running devkitPro on VirtualBox with Windows XP) and the results weren't better.
_________________
Sh*tty RPG

#164551 - Cydrak - Fri Nov 07, 2008 1:18 am

Quote:
> how big is said assembler file?
About 30 MB.
o_o;
Quote:
I don't understand why an assembler can be so slow. ...
30MB / 80 chars per line / 5 min, roughly speaking, would be ~1,300 lines/second. Take that as you will.

Frankly I'd be surprised (or impressed) if it wasn't a little slow. Traditionally these tools work on human-written code, after all. They are a bit fancier for it. You could even say, there are practically infinite ways to write a given program but roughly one way to write a MOD; by comparison, MODs have a very fixed, simple format. (Well... that's giving the latter too much credit, IMHO, but I digress. ;p)
Quote:
arm-eabi-gcc -MMD -MP -MF /c/ProyectoTSC/build/AAS_Data.d -x assembler-with-cpp -mthumb-interwork -mthumb -c AAS_Data.s -o AAS_Data.o
This doesn't just invoke the assembler; it also runs the C preprocessor ("cpp") beforehand! So you really have two programs involved. You can further try it without "-MMD -MP -MF AAS_Data.d", which is superfluous for simple data. I assume that's the dependency search mentioned above, and I doubt it works without the preprocessor anyway.

If AAS allows you to convert to binary (and separate files), perhaps look into something like GBFS, which is designed for embedding this stuff. It sidesteps the extra compilation entirely.

#164552 - Chano Marrano - Fri Nov 07, 2008 2:31 am

Quote:
You can further try it without "-MMD -MP -MF AAS_Data.d", which is superfluous for simple data. I assume that's the dependency search mentioned above, and I doubt it works without the preprocessor anyway.
I tried to execute this with the same result:
Code:
arm-eabi-as.exe C:\...\AAS_Data.s -o C:\...\AAS_Data.o


Quote:
If AAS allows you to convert to binary (and separate files), perhaps look into something like GBFS, which is designed for embedding this stuff. It sidesteps the extra compilation entirely.
That's the problem, AAS converter only gives as output a single assembler file for all the mods converted, because it tries to share the samples used in all songs. AAS converter takes as input a directory instead of a file.

Quote:
Frankly I'd be surprised (or impressed) if it wasn't a little slow.
Five minutes for a 30MB file?
_________________
Sh*tty RPG

#164574 - Miked0801 - Fri Nov 07, 2008 7:11 pm

What CPU/Memory is your computer? Perhaps the thing is running full speed. 30Megs of dense assembler is a bunch of work. If you have a multi-cpu system, would it be possible to divide and conquer the assmebr file?

#164576 - Chano Marrano - Fri Nov 07, 2008 9:34 pm

I have a AMD X2 3800+ with 2GB, but as I said I'm running Windows XP through VirtualBox. AAS converter takes less than a minute to get its job done, so I don't think a better PC would fix my problem.

VirtualBox doesn't allow dual core guest systems.
_________________
Sh*tty RPG

#164579 - Cydrak - Sat Nov 08, 2008 1:09 am

Quote:
Five minutes for a 30MB file?
Even with GCC out of the picture? I still feel sorry for the assembler, but fair enough!

I tried and couldn't reproduce this, in or out of VirtualBox. I'm running XP on an X2 5200+, 3GB RAM; VBox had 1GB. Data was a 16MB image exported from Usenti, and AS takes ~3 seconds on the 60MB file. I have to admit, this isn't so bad as I thought. Of course my data and host are different, and it was almost a bare VM image, so this isn't saying too much. :|

One oddity for me is that files under //vboxsvr/* take a little longer (sometimes), and 'gcc -x assembler-with-cpp' even segfaults on them, whereas the assembler alone works fine.

The worst I could manage: GCC on the equivalent C array spends about a minute. (Assuming RAM is available, because it tops out at a monstrous 1GB. Yikes!)

#164583 - Chano Marrano - Sat Nov 08, 2008 8:28 am

Hmmm... OK, project link.
All mods have been downloaded from modarchive.org and www.exotica.org.uk . If you want to compile the project, you'll need to set TONCCODE and LIBAAS variables in Makefile.
_________________
Sh*tty RPG

#164605 - Cydrak - Sun Nov 09, 2008 7:40 pm

...several minutes. Yep. @_@;

AS kinda doesn't like long lines. Most of AAS's assembler is just one statement, a 30MB .byte array. I've seen mammoth text files, but never lines this long! Pretty egregious abuse if you ask me...

I think I would be looking for another library. ;p But if you must have AAS, a slightly easier solution--than rewriting the converter--is to break up the lines. I say "slightly" because the first shell commands I tried fare even worse than the assembler. >_>

So as a starting point I made a very quick-and-dirty C hack, to split all those huge arrays: AAS_Linefix
Add it to the conversion rule:
Code:
AAS_Data.s: $(AAS_DEPS)
    $(LIBAAS)/bin/conv2aas.exe $(AAS_DIR)
    $(SOME_PATH_TO)/AAS_Linefix.exe AAS_Data.s

If it works for you, things should be considerably faster now. Good luck!

#164607 - Chano Marrano - Sun Nov 09, 2008 9:11 pm

o_O'

WOW

Uh... thanks!
A explanation for the problem would have been more than sufficient for me, so the fact that you have spent your time writing a program to fix it has left me speechless.

By the way, as.exe now gets its job done in less than 20 seconds, so again, thanks a lot!
_________________
Sh*tty RPG