Unicon / lsof reporting bug

[Anonymous]

There seems to be a discrepency - on my machine at least - between files in use under iconx and those reported by lsof. I don't know how the interaction between the two works, but I would've expected lsof to get the info from the OS, so I'm not sure what's going on here. I don't want to post it as a unicon bug if it's lsof, and vice versa, so any help pinning this down would be appreciated.

I was gettng "open files" errors under linux due to extremely lazy and inefficient coding on my part - I'll redo this later - and was using lsof to look at what was going on.

The code basically looked like this (with functions omitted):

        global merged_file

procedure main()

        merged_file := open("test_merged.csv", "w") | stop(&errortext, " merged_file")

        DoFileMerge()

        close(merged_file)
end

procedure DoFileMerge()

        main_file := open("test_main.csv", "r") | stop(&errortext, " main_file")

        while EachLineOfMain(read(main_file))

        close(main_file)
end

procedure EachLineOfMain(main_line)

        steamer_file := open("test_steamers.csv", "r") | stop(&errortext, " steamer_file")
        drifter_file := open("test_drifters.csv", "r") | stop(&errortext, " drifter_file")

        steamer_list := []
        while line := read(steamer_file) do put(steamer_list, line)
        drifter_list := []
#while line := read(drifter_file) do put(drifter_list, line)

        close(steamer_file, drifter_file)
#close(steamer_file)

        while every WriteIfFound(main_line, !steamer_list)
#while every WriteIfFound(main_line, !drifter_list)

        return
end

I had commented out drifter_file usage as that was where the open files error was apparently being reached.
Running the program however had lsof reporting lots of open files - in drifter_file! If I exclude the open & close for drifter_file also, lsof does not report that steamer_file is in use, even though it obviously is...

Is this likely to be a unicon issue, or lsof?

[Anonymous]

To add some info to this query, I have discovered that the problem isn't with lsof per se.

If I look at /proc/<pid>/fd for the running unicon process, this also appears to list the files incorrectly. There are many drifter_file entries showing, and one for steamer_file (although the steamer_file entry doesn't always appear!); I would expect either one for each, or many for each (depending on when Unicon releases the resources).</pid>

I'm guessing no-one else has seen anything like this?

[Anonymous]

To make an attempt at finding the cause, I rewrote the above section of code in Python, and running lsof while the process runs shows just one entry for each of the four files, as I originally expected.

As I'm new to Unicon, I wondered if part of the cause of the issue was due to the return statement in EachLineOfMain? I added this as the while statement running EachLineOfMain would complete after only one file line without it.
Is that somehow causing Unicon not to tidy up after itself, or am I doing something else wrong to stop Garbage Collection working properly?

[Jafar]

Can you please provide a mimial example to reproduce this issue, with the expected results and the actual results?

Thanks!

[Anonymous]

Hi Jafar.

Minimal python program:

import time

def main():

    for eachNum in range(10000):
        eachLineOfMain()

def eachLineOfMain():

    steam_file = open('test_steamers.csv', 'r')
    drift_file = open('test_drifters.csv', 'r')

    time.sleep(0.5)

    steam_file.close()
    drift_file.close()

main()

Waiting approx 5 secs then using:

lsof -wF n | grep test_

Produces:

n/home/graham/Documents/Python/test_steamers.csv
n/home/graham/Documents/Python/test_drifters.csv

i.e. one entry per file.

An equivalent unicon program:

::unicon
procedure main()

    every 1 to 10000 do eachLineOfMain()

end

procedure eachLineOfMain()

    steamer_file := open("test_steamers.csv", "r") | stop(&errortext, " steamer_file")
    drifter_file := open("test_drifters.csv", "r") | stop(&errortext, " drifter_file")

    delay(500)

    close(steamer_file, drifter_file)

end

Produces an lsof output of:

n/home/graham/Documents/Unicon/Programs/test_steamers.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv
n/home/graham/Documents/Unicon/Programs/test_drifters.csv

So it looks as though unicon is releasing the first file, but not the second before repeating the loop.

---

mod necro-edit for the syntax colouring, as of 2020

[Jafar]

Now that it is short I can follow it :-).

the issue is that you are passing two files to one call close(). What makes you think you can do that? Do we have something wrong in the documantation that suggets that?

make two seperate calls to close() and that should fix it. Extra parameters to fucntions are usually ignored. So the second file was just being ignored here. If you prefere one call to close() you can do:

every close(f 1| f2)

[Edit] of course, this is still two calls to close(), but it allows to be more concise sometimes.

[Jafar]

BTW, if you login to SF then you can post directly without having your posts going through admin moderation and slowing them down sometimes.

[Anonymous]

Thanks, Jafar. Not sure where I got that idea from...!

I started posting on a machine that wasn't logged into SF, and thought it would confuse matters if I suddenly started posting under a different ID - sorry.