Saturday, June 3, 2017

NSF Merit Review Process

It has been a long time. I moved to UCF as of August 2016 and have been re-establishing my research group. We are moving on with more research.

Today's blog topic is somewhat ad hoc. I just submitted my feedback for the National Science Foundation (NSF)'s Merit Review process per NSF's request, and thought I should write to my blog about it. First, let me say that submitting proposals to NSF and participating in the merit review process as a panelist is one of the most enjoyable and rewarding activities you can do as an academic. It is, by far, the most fair and the most thorough system for funding fundamental research. Yet, I have to admit I have not seen the practices of all agencies, e.g., NIH. So, my comments below are meant for making minor improvements to an already excellent system.

If you are not familiar with the NSF proposal merit review process, there are several online resources or experiences you can read to have a good understanding of it, e.g.:
https://www.nsf.gov/bfa/dias/policy/outreach/grantsconf/meritreview_feb16.pdf
http://www.sciencemag.org/careers/2003/04/nsf-grant-reviewer-tells-all
http://www.astrobetter.com/blog/2013/07/08/the-inside-scoop-on-nsf-review-panels
https://web.engr.oregonstate.edu/~grimmc/NSF/TheNSFReviewingProcess.html

Here are some ad hoc thoughts and suggestions on improving the NSF proposal review process:

15-Page Project Description: 15 page limit is too long. As proposers, we naturally tend to fill up all the 15-page space to have a better chance in winning against competitor proposals. But, 15-pages is too long. We end up spending too much time writing proposals, and spend increasingly less time doing research. As it is in the panels, most reviewers don't read the details, which is unfortunate but the reality. So, the proposers end up writing 15 pages of a document which is not fully read by the reviewers. I think 6 pages for the technical content and 2 pages for the broader impact (including the educational component as well as prior NSF support) would be very much enough to lay down a high risk high reward research agenda. The reviewers will then be more inclined to reading everything in the proposal.

Formality (or Balloon) Documents or Content: We have totally unnecessary formality documents or content in the full proposal submissions. For instance, Data Management Plan and Postdoc Mentoring Plan rarely make any difference in the decision making process. If these are needed for regulatory purposes, they can be asked for when a particular proposal is picked for funding -- this would reduce a lot of paperwork considering the fact that 80+% of the proposals don't get funded anyway.

Obsolete Panel Review System at Fastlane: I think an upgrade (or overhaul) of the Panel Review System is way overdue. We, as panelists, are still wasting our time on fixing an apostrophe when copying our reviews into the Panel Review System. Instead of using modern features like tracing of changes (as in MS Word) or suggesting of changes (as in Google doc), we are still fixing the writing errors via sending a comment to the scribe or simply shouting across the table "Hey John, second paragraph third sentence in the panel summary: Please fix the typo 'covfefe' !"  Hopefully the scribe is not going to get offended with all this in the mean time. If the typo is in a panelist's review, other panelists just don't bother to avoid any possibility of offending someone who may later be reviewing their proposals! Maybe too fancy, but I wish there was an anonymous commenting/suggesting feature that other panelists can use to give feedback to other panelists. Yes, anonymity is good -- even within a panel. :)

Panelist Honorarium Amount: Currently, if a panelist is attending in person to a 2-day panel, (s)he gets about $1,200 for covering expenses. Considering all the time needed for reading the proposals and writing the reviews, and the travel hassle one has to go through to attend the panel, I think this is too little. This may be one reason why panelists have increasingly been less careful in writing a quality review. If this amount is not increased to something significant, I think it will be hard to find panelists.

1099 Form for the Panelist Honorarium: After a panel, panelists receive a 1099 form for tax reporting of the honorarium (or whatever it is supposed to be called). The problem is that after entering this form in the tax return, it increases the total taxable income and the panelist ends up paying "self employment" tax. I am not sure, but, depending on your tax bracket, attending a panel may cause a loss rather than additional income! In one case I remember paying "alternative minimum tax" due to the self employment income from the panels. Accounting-wise this may be a stupid question to ask, but can't NSF make the honorarium non-taxable? Let alone the additional tax you have to pay due to attending a panel, having not to deal with entering the 1099 form when filing taxes would be a relief.

Friday, January 16, 2015

Device-to-Device (D2D) Communications: Why do we need it for public safety?

D2D communication means that devices talk to each other to garner opportunities to perform end-to-end communications or transfers. A typical and old example people use to describe D2D is that why do our mobile phones have to synchronize and communicate with the base station if they are very close to each other, perhaps in the same room. The D2D argument has been to exploit such situations and harvest more out of the scarce spectrum by not bothering the spectrum with connections to the base station(s).

The D2D argument goes even further.. What if our devices can do this opportunistic talking to each other over multiple hops. That will extend the reach of the base stations (and hence the Internet), right? Yes, this is true. However, not so easy. Many innovations across the protocol stack as well as hardware front end are needed. In our recently funded NSF project, we are taking a stab at research challenges involved in reaching such an environment where devices talk to each other, and further, share their wireless connectivity/resources pervasively: https://sites.google.com/site/pss4psc

Like most other NSF projects, the concept of "pervasive sharing" is risky and draws easy criticism from our peers. Why would providers be willing to share their preciously licensed spectrum bands with the customers of other providers? Why would a user be incentivized for allowing his/her phone to relay traffic for others, which clearly involves privacy risks and exposure to more cyber-attacks? How would you handle the heterogeneity of the wireless connections? -- which I call "substrates" to make up something larger and good. The questions go on and on..

I guess one simple answer is the "larger good"! If there is a larger good that threatens everybody, then, history shows, people are motivated for sharing. Sharing and cooperation happen more when the fate of the whole system or the ship is in danger. Public safety is an example case.. Safety of the public and the people is important and everybody, eventually, cares about it. I think notions like pervasive sharing or D2D make sense for these types of larger goods.

Further, I do think that, given the literally exponential growth (I don't use the term "exponential" to attract attention, but use it carefully.) of mobile data demand, our wireless communication infrastructure is in danger. We, as the whole community, need to find a solution and cooperate. Cut-throat competition when improving a system's performance helps until a point. But, beyond a point, just like thrashing in operating systems' multi-programming level (a textbook classic), competition degrades what we can collectively achieve. That is the point when regulations incentivizing sharing are needed.

Folks have recognized this need for sharing and cooperation in spectrum management a while back.. Tethering is now part of almost all phones, which allow sharing of data plans with devices nearby. Going further, opengarden.com designed an app that allows mostly peer-to-peer sharing of wireless connectivity among devices. The power of cyber-sharing and openness is unprecedented. Yet, achieving it as a profitable business/system is hard.

Yet another case where D2D communication and sharing makes sense is disastrous scenarios. Here is a recent incident showing how D2D could have helped responding to an emergency in metro: http://www.firefighterclosecalls.com/news/fullstory/newsid/216435  We had reports indicating that crowdsourcing via D2D communications helped catch Boston Marathon: http://www.cnn.com/2013/04/24/opinion/kessler-digital-forensics

More could be done for public safety by employing D2D communications -- perhaps catch the bombers even before they detonate the bombs. Establishing common sense is what we need to do. But, that common sense requires more seamless D2D and more sharing.. To the level that it may have to be pervasive.

Sunday, December 7, 2014

A spell check could change many things. Just use meta-writing tools!

I just finished reviewing a writeup. It was full of spelling errors, sloppy sentences, and annoying spacing and syntax errors. Flat out ugly! It is simply not in the best interest of you to send a colleague hastily written stuff. This is not just true for "official" writing like memos, papers or theses; but also true even for your emails or instant messages -- IMHO. Here are some of the reasons:

  1. (First) Impression is Almost Everything: This has become a rule of thumb for the modern society, but, it seems that we just forget it. When a reviewer reads a sloppily written document, the likelihood that (s)he giving a positive rating about your document is very close to zero. 
  2. Your Writing is Who You Are Inside: What you write eventually originates from your who you are inside. If you are an organized well-disciplined person, this will reflect on your writing too. Writings of people who think clearly tend to have clarity and brevity -- both good features of writing.
  3. Your Writing is Your Signature: What you write is legal and can be used for and against you! Even emails and instant messages are like that too. That's why the term "documented" is used for an evidence going in record. No sane person would want to be associated with an illegal piece of written material. 
  4. I Write Therefore I Exist: In academia, writing is so important that without your written material you simply don't exist. No academic would be known if (s)he does not publish something written. Yet, you are associated with what you write. You don't want to be known/associated with badly written material, do you?
  5. Annoying, Annoying, Annoying: It is so annoying to spend time on a colleague's obvious writing errors. A colleague/collaborator is not there to correct your writing errors, but rather to have an intelligent communication on ideas and insights. (S)he may help in rephrasing your sentences to express the ideas/insights better, not in correcting them.

All of the above are the reasons why a writer must be extra careful. So, what is the solution? How can we make sure our colleagues don't get annoyed or we don't put ourselves into a bad situation because of a writing error? Well, there is no 100% avoidance strategy for this. You just try to be careful or freaked about what you write to people. But, meta-writing tools help a lot. Here are some suggestions:

  • Spelling Errors: These are the most obvious ones to fix. Just use a spell checker for God's sakes! Virtually every editor has such checkers. It is a shame to ignore such spell checkers and send a document to someone before doing a spell check.
  • Formatting Errors: Go through every page of your document and look for obvious formatting errors. Things like figures/text going out of margin, blur figures, something not appearing properly, etc. Have somebody close to you review it quickly. Even your child or spouse could do it. If something is not appearing right, another eye should be able to see it quickly. It does not have to be an expert eye who understands the written material. 
  • Grammatical Errors: If you cannot write well in English, use a professional editor to get your document corrected. And, don't think of the money you spend on this as a waste. It is actually money well spent. Also, in most cases, large institutions have support system (e.g., a writing center) to review and correct/improve your writing. Just get help from them. Either way, getting somebody else to help typically means that you will have to prepare your writeup well ahead of time to make the time for reviewing/improving.

Remember it is your document. You are the person writing it. So, spending an extra hour before sending it to others is not a luxury, it is a requirement for interacting with others.

Monday, November 17, 2014

Why meta-X?

Hello colleagues, friends, foes..

I intend to maintain this blog as much as I can and sort of share my feelings and thoughts, and hopefully facts too. I do want to keep the blog scoped to my work, which involves research and teaching computer networking.

I started with the blog name "meta-networking" but I/we may change it as time goes by. I did partially copycat the name from Prof. Demirbas's blog: http://muratbuffalo.blogspot.com. Meta-X generally refers to something that helps X be X. Sometimes "meta" might be beyond just help, it might be necessary to make X. Meta-networking, in that sense, means the stuff you do to keep the network up and running. Protocols, money, labor, human behavior and other stuff I may be forgetting now all count into meta-networking.

Meta-X also has this other meaning that meta is the residual or sort of like a dual of X. You maintain the meta along with X and use it for "helping" X be X. So, this meaning of meta is more recessive rather than being necessary for X. For that, meta-networking means everything you do for managing the network better. You simulate the network in parallel to the online running network -- a concept we have been exploring in our recent NSF project: http://www.cse.unr.edu/~yuksem/omega.htm. The real thing is already running, but you do this meta stuff to *help* it run better. I admit it is somewhat hard to explain the difference between the former meta and latter here. But, I feel the difference, trust me. :)

Perhaps the earliest and most-known meta-X was meta-programming. Back in 1990s (gosh I am getting old), we were trying to get the C++ compiler do weird stuff while compiling the code. The compiler would print warning/error messages that had some (funny) meaning! Why in the world would you do that, right? But, it was a major "concept" for the advanced programming course in the graduate studies at CS@RPI. I remember using templates and their specializations to make the compiler embark a recursive instantiation of a class and print funny/exciting messages by placing some type-mismatch in an assignment statement in each instantiation. The compiler would go into infinite recursion while trying to instantiate a class! Thankfully, prog languages people got better use of meta-programming and came up with some real application of the concept of doing some meta stuff that does consider your program code as "data": http://en.wikipedia.org/wiki/Metaprogramming. I am glad to see that all those compiler hacks are put in past. I really don't want to deal with any meta-IF or meta-WHILE statements after the age of 40.

Well, I guess this is enough for the first post. I made enough of a fuss about "meta".